r  AD-A104  593  NAVAL  POSTGRADUATE  SCHOOL  MONTEREY  CA  F/G  5/1 

LEADERSHIP  AND  MANAGEMENT  EDUCATION  AND  TRAINING  (LMET)  EFFECTI— ETC<U) 
JUN  81  D  L  VANDOVER»  J  P  VILLAROSA 


UNCLASSIFIED 


NL 


Unclassifi ed 


security  classification  a*  this  naoi  rm >«■  o mm  *nmm« 


REPORT  DOCUMENTATION  PACE 


READ  INSTRUCTIONS 

BEFORE  COMF1.  ETtN  (j  FORM 


NT'S  catalog  NUMRCA 


TITlC  <mtm  JwAflfJ 


Leadership  and  Management  Education  and  Training 
-  ((.MET)  Effectiveness:  A  Pilot  Study  for  r  ~ 
Evaluation  „ 


David  Lee /Vandover  ami  John  Poteat/Villarosa 


NSRNORMINR  ONO.  NENONT  NUMRCN 


».  NERNORMINO  ONOANlZ  ATlON  NAMS  ANO  AOONCSI 

Naval  Postgraduate  School 
Monterey,  California  93940 


II  CON  TNOLLIMO  OFFICE  NANI  ANO  ADDRESS 

Naval  Postgraduate  School 
Monterey,  California  93940 


IfP^ 


14  MONITORING  AGCmCy  NAME  4  AOOMfStSff  ihffnml  tmm  CmmtrmtlHtg  Otti**)  I  IS.  SCCUftITV  CLASS,  ft  thim  r*frt) 


Naval  Postgraduate  School 
Monterey,  California  939 


Unclassified 


IS.  DISTRIBUTION  STATEMENT  faf  INI*  Rnnii 


Approved  for  public  release;  distribution  unlimited. 


IT.  OISTNIRUTION  STATEMENT  f.l  IA«  aaairact  M«N  m  IlMl  M.  If  NfffMRI  Ann.  RnmI) 


It  KCV  WONOS  IXanIIINM  AN  f.N»M  «M  II  HNIIWf  >N  IRMIfft  *r  M««*  lf*a> 


Leadership  and  Management  Education  and  Training  [LMET],  Pilot  Study, 
Interviewing,  Effectiveness,  Leadership  Management  Training,  Program 
Evaluation. 


x  /  _  _  _ _ _ 


10.  ASIH^CT  (Cmmiimm  «n  nvrn  Ntft  If  WWfR  ifmiitf  If  UN#  MM) 

This  report  describes  the  design  and  use  of  the  pilot  study  concept  as  a 
preliminary  step  in  directing  Navy  program  evaluations.  This  methodology 
focused  specifically  upon  the  Navy  Leadership  and  Management  Education  and 
Training  Program  [LMET]  to  identify  relevant  effectiveness  issues  to  provide 
program  managers  necessary  guidance  for  overall  program  evaluation.  Interviews 
of  a  cross-section  of  51  LMET  graduates,  their  immediate  supervisor  and  sub¬ 
ordinate  were  conducted  in  an  effort  to  determine  leadership/management 


30  i  jam  tj  1473  EDITION  ON  I  NOV  I*  ORROLBTE 
(Page  1)  S/N 


Unclassified  ml  ^  -*■ 

SECURITY  CL  ARM  NIC  ATlON  ON  TmIR  RARE 


2$iM30 


i 


grrerrxmnira  onancn  rsrazn 


imorovement.  Results  indicate  specific  recommendations  concerning;  the 
use  of  pilot  studies,  the  interviewing  process  and  LMET  evaluation. 


*J  t 


‘orm  1473 


Unclassified 


Approved  for  public  release;  distribution  unlimited 


Leadership  and  Management  Education  and  Training  [LMET]  Effectiveness: 

A  Pilot  Study  for  Evaluation 


Authors: 


Approved  by: 


David  L.  Vandover 

Lieutenant  Commander,  United  States  Navy 
B.S.,  United  States  Naval  Academy,  1971 


John  P.  Villarosa 

Lieutenant  Commander,  United  States  Navy 
B.S.,  University  of  North  Carolina,  1971 


Submitted  in  partial  fulfillment  of  the 
requirements  for  the  degree  of 


MASTER  OF  SCIENCE  IN  MANAGEMENT 


from  the 


NAVAL  POSTGRADUATE  SCHOOL 
June  1981 


T" 


iesi$  Advisor 


>cond  Reader 


lalrman,' 


LmenTS  of  Administrative  sciences 


Ay  M 


»an  of  information  and  Policy  sciences 


mm- 


ABSTRACT 


This  report  describes  the  design  and  use  of  the  pilot  study 
concept  as  a  preliminary  step  in  directing  Navy  program  evaluations. 
This  methodology  focused  specifically  upon  the  Navy  Leadership  and 
Management  Education  and  Training  Program  [LMET]  to  identify  relevant 
effectiveness  issues  to  provide  program  manaqers  necessary  guidance  for 
overall  program  evaluation.  Interviews  of  a  cross-section  of  51  LMET 
graduates,  their  immediate  supervisor  and  subordinate  were  conducted  in 
an  effort  to  determine  leadership/management  improvement.  Results 
indicate  specific  recommendations  concerning;  the  use  of  pilot  studies, 
the  interviewing  process  and  LMET  evaluation. 
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I.  INTRODUCTION 


In  late  1978,  the  U.  S.  Navy  adopted  a  massive  training  program  for 

its  personnel  in  leadership  and  management.  The  program  is  planned  to 

systematical ly  replaced  all  existing  leadership-management  courses  of 

instruction.  Its  implementation  was  Navy-wide  with  first  priority  for 

school  quotas  given  to  personnel  assigned  to  operational  fleet  units. 

As  the  only  leadership-management  program  in  the  Navy,  it  was  large  and 

applied  to  all  Naval  levels  of  command.  Due  to  its  widespread 

application,  the  program  required  large  resource  expenditures  in  time, 

manpower  and  money.  Manpower  requirements  were  expended  in  students 

attending  the  ten  day  course  as  well  as  instructor  and  administrative 

time.  The  overall  goals  of  the  program  known  as  Leadership  and 

Management  Educating  and  Training  [LMET]  were: 

To  provide  a  formal  and  systematic  training  program  for 
professional  development  of  Navy  leaders  at  critical  points 
in  their  careers,  based  on  research  of  effective  Navy  leader¬ 
ship. 

To  train  officers  and  petty  officers  in  the  specific  leader¬ 
ship  and  management  skills  needed  to  perform  effectively  at 
their  level  in  the  chain  of  command. 

To  conduct  ongoing  evaluation  for  improving  and  updating  these 
programs . 

To  encourage  Navy  leaders  to  take  personal  responsibility  for 
implementing  effective  leadership  skills,  by  means  of  an  edu¬ 
cational  approach  that  emphasizes  individual  initiative  and 
accountability  for  effective  performance  as  a  Navy  leader. 
[HRMC.N.O. ,p .7] 

The  cost  of  the  program  was  to  be  offset  through  "increasing  the  Navy's 
ability  to  achieve  its  overall  mission  by  increasing  the  effectiveness 

n 


of  Navy  leadership  across  all  levels  of  the  chain  of  command." 
[CNET,N.D. ,p.5] 

The  Navy  has,  and  continues  to  invest  time,  manpower  resources  and 
money  into  this  program.  Yet,  there  has  been  to  date  little  effort  de¬ 
voted  to  evaluation  of  its  effectiveness  or  results.  The  question  of 
where  does  the  program  stand  today,  remains  unanswered.  Recent 
comments  of  LMET  instructors  concerning  the  course  vary,  typically: 

The  course  if  not  fun  to  teach . the  [teacher]  burnout 

ratio  is  high. 

It  will  raise  the  average  guy  or  top  performer _ it  won't 

work  for  the  below  average  performer,  we've  lost  seme. 

It  is  a  seed  planting  evolution  with  attempts  at  behaviorial 
change . 

You  have  to  sacrifice  one  [student]  for  the  good  learning  of 
twenty-eight  others. 

You  can't  tell  a  student  he  is  wrong ... .[You  can]  hope  to 
make  him  see  what  his  affect  is  on  others. 

Additionally,  the  question  arises,  what  is  the  perceived  value  of  the 

LMET  Program  to  managers  and  leaders?  Recently  interviewed  graduates 

of  the  LMET  course  had  these  remarks  concerning  their  training: 

Before  LMET  I  didn't  know  which  way  to  go . LMET  produces 

immediate  results . it  really  helped  me. 

LMET  probably  helped  this  guy  more  then  any  other  professional 
school  in  his  career . 

If  the  decision  was  mine  I  wouldn't  send  anybody  to  it  [LMET 
School] . 

It  [LMET  School]  showed  me  things  that  were  not  clear  be¬ 
fore . it  helped  me  to  understand  myself. 

If  the  command  doesn't  support  the  training  [LMET  School]  the 

man  receives  then  I  wouldn't  send  anyone  else . that  is 

more  detremental  than  not  sending  the  guy  at  all. 
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Clearly,  there  is  disagreement  anong  those  teaching  and  implementing,  as 
well  as  those  receiving  the  training. 

The  Navy  in  the  late  1960's  and  early  1 970 1  s  was  confronted  by  a 
number  of  contemporary  problems  that  were  indicative  of  the  turmoil 
within  society  as  a  whole.  These  problems  included  drug  and  alcohol 
abuse,  high  rates  of  absenteeism,  low  retention  rates  of  skilled  tech¬ 
nicians,  high  attrition  of  newly  recruited  personnel,  and  an  atmosphere 
of  crisis  management.  It  was  difficult  to  link  any  given  problem  to  a 
simple  cause  and  effect  relationship.  Rather,  the  problems  appeared  to 
be  intertwined  around  the  issue  of  ineffective  leadership  and  manage¬ 
ment  at  all  levels  within  the  chain-of-command.  It  was  imperative  that 
action  be  taken  to  reverse  this  situation  and  it  was  to  that  end  that  the 
Navy  turned  to  the  LMET  Program. 

It  is  now  mid-1981  with  the  LMET  Program  ending  its  third  year.  As 
with  any  large  program,  LMET  was  designed  to  achieve  certain  goals  while 
solving  specific  problems.  In  order  to  determine  if  LMET  has  done  this, 
that  is  to  judge  its  effectiveness,  it  is  necessary  to  evaluate  it.  Such 
an  evaluation  would  determine  if  the  program  is  accomplishing  its  stated 
goals.  In  fact,  one  of  the  goals  of  the  program  specifically  called  for 
ongoing  evaluation  and  updating.  [CNET,N.D.]  The  developer  of  the 
program,  McBer  and  Company,  also  recommended  an  on  going  evaluation.  With 
an  evaluation.  Navy  leaders  could  assess  its  health  thereby  assuring  that 
it  remain  an  ongoing  program,  meeting  its  goals  within  the  Navy.  This 
evaluation  is  especially  important  to  program  managers  to  enable  them  to 
judge  its  cost  effectiveness  during  these  times  of  austere  funding.  De¬ 
termining  whether  the  LMET  Program  is  "increasing  the  effectiveness  of 
Navy  leaders  is,  however,  no  simple  task. 
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Evaluation  of  any  large.  Navy-wide  program,  such  as  LMET,  is  in 
itself  an  expensive  undertaking  in  terms  of  budgetary  considerations  as 
well  as  time  and  manpower  requirements.  For  this  reason  the  results  must 
be  valid  and  relevant  to  the  program  being  evaluated.  Before  embarking 
upon  such  a  task  as  program  evaluation,  decision  makers  need  some 
assurance  that  the  evaluation  will  in  fact  provide  relevant  data  about 
the  program  under  examination.  In  addition,  the  necessity  to  conduct  an 
efficient  yet  thorough  evaluation  further  compounds  the  decision  maker's 
dilemma.  One  method  of  beginning  such  an  evaluation  is  through  the  use 
of  a  pilot  study.  The  concept  is  basically  for  an  organization,  such  as 
the  Navy,  to  use  some  of  its  own  assets  to  conduct  a  mini-evaluation  of 
the  program.  This  Pilot  Study  could  test  the  hypothesis,  questions,  and 
methodology  with  which  a  larger,  full  scaled  evaluation  may  be  conducted. 
In  evaluating  the  pilot  study  outcome,  revisions  or  changes  can  be  made 
prior  to  commencing  the  much  costlier  evaluation. 

This  then  is  the  focus  of  this  research,  to  conduct  a  pilot  study 
of  the  Navy's  LMET  Program,  thereby  assisting  in  the  development  of  a 
larger,  full  scale  evaluation.  The  pilot  study  was  undertaken  by  two 
Naval  officer  graduate  students  using  a  design  technique  and  strategy 
developed  with  the  assistance  of  McBer  and  Company  and  past  experience. 
Data  from  the  study  was  used  to  provide  a  rough,  first-cut  evaluation  of 
the  LMET  Program  and  more  importantly,  an  evaluation  of  the  methodology 
used.  This  information  can  then  be  applied  by  the  sponsor  of  this  study 
to  design  a  full  scale  evaluation  of  the  LMET  Program. 


The  thesis  is  divided  into  six  major  chapters;  the  first  chapter  is 
the  introduction.  The  second  is  a  brief  history  of  the  evolution  of 
leadership  philosophy  and  development  within  the  Navy,  with  specific 
background  on  the  development  of  the  Navy  LMET  Program.  Emphasis  is 
placed  on  Dr.  D.  McClelland  and  McBer  and  Company's  concepts  on  motive 
acquisition  and  competency  identification.  The  LMET  Program  is  detailed 
with  additional  thoughts  offered  on  the  need  for  its  evaluation.  The 
third  chapter  discusses  those  issues  and  problems  anticipated  in  the 
design  of  the  pilot  study  itself  including  the  methodology  that  was  used. 
Next  is  a  description  of  the  actual  pilot  study  that  was  conducted, 
including  a  discussion  on  preparations  for  data  gathering, 
pre-arrangements  conducted  with  respondents  and  the  method  used  to 
analyze  data.  In  simple  terms  it  is  what  happened  and  when,  during  the 
pilot  study.  The  fifth  chapter  is  the  analysis  and  findings  of  the  Pilot 
Study  research.  It  is  divided  into  two  major  areas,  that  of  the 
interview  methodology  [process]  and  a  rough  analysis  of  the  data 
[content]  as  it  applies  to  LMET  specifically.  The  final  chapter  is  an 
assessment  of  the  Pilot  Study  concept,  with  recommendations  directed 
toward  a  full  scale  LMET  evaluation. 


II.  HISTORICAL  BACKGROUND  OF  NAVY  LEADERSHIP  AND  TRAINING 


A.  HISTORICAL  REVIEW  OF  NAVY  LEADERSHIP 

Since  the  birth  of  the  United  States  Navy,  senior  officers  have 
recognized  the  ability  of  certain  skilled  leaders  to  motivate  subordin¬ 
ates  in  achieving  seemingly  unattainable  goals.  These  were  the  leaders 
who  were  called  upon  in  time  of  need  for  accomplishment  of  those  tasks  to 
which  others  had  failed.  They  were  the  ones  called  upon  when  certainty 
of  the  outcome  was  more  than  desirable,  but  rather  a  necessity.  Using 
these  leaders  as  shining  examples  of  what  a  good  Naval  leader  should  be, 
the  Navy  ingrained  in  its  leaders  the  “tradition"  of  outstanding 
leadership,  laced  with  names  such  as  John  Paul  Jones,  Truxtun, 
Bainbridge,  Porter,  Perry,  Farragut,  Taussig,  Halsey  and  Nimitz.  In 
order  to  be  a  really  great  leader  one  had  only  to  emulate  these  fine 
examples,  the  Great  Man  concept.  The  traits  that  great  leaders  viewed  as 
being  desirable  were  obviously  the  traits  that  would  create  good  leaders. 
Hence,  John  Paul  Jones'  thoughts  in  "Code  of  a  Naval  Officer",  were 
memorized  by  midshipman  as  the  proper  conduct  for  a  Naval  officer. 
Pleble's  emphasis  on  discipline  and  drilling  was  the  gospel  on  the 
training  of  subordinates,  while  Farragut’s  disregard  of  "Torpedoes"  was 
an  example  of  leader  courage.  Additionally,  what  was  left  unsaid  but 
amply  demonstrated,  was  that  good  leaders  would  inevitably  become  famous 
for  their  abilities.  Hence,  an  end  means  inversion  occured  developing 
the  notion  that,  if  one  were  well  known  or  famous,  then  one  must  surely 
be  a  great  leader. 
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Until  World  War  II,  this  idea  of  a  tradition  bound  leader  was  the 
most  prevelant  view  held  throughout  the  Navy.  During  WW  II  with  the 
large  influx  of  personnel  due  to  a  quickly  expanding  Navy,  leaders  not 
ingrained  in  tradition  were  introduced.  This  large  influx  of  leaders  at 
all  levels  could  not  be  assimilated  into  this  tradition  bound  leadership 
style.  Success  or  failure  of  their  personal  leadership  styles  were  based 
largely  on  the  end  results  achieved,  which  may  or  may  not  have  been 
steeped  in  tradition.  Since  on-the-job  leadership  training  was  not 
practical,  it  was  viewed  that  some  rudimentary  training  in  leadership 
could  avoid  the  pitfalls  of  this  trail  and  error  method  of  leadership. 
Leadership  training  would  improve  the  performance  of  those  born  leaders 
as  well  as  those  less  capable  personnel  occupying  leadership  positions. 

Since  World  War  II,  the  Navy  has  emphasized  its  training  at  the  be¬ 
ginning  of  a  person's  military  service  with  such  programs  as  boot  camp. 
Officer  Candidate  School,  and  "Plebe"  summer  at  the  Naval  Academy. 
Occupational  training  is  generally  given  immediately  after  boot  camp  and 
prior  to  the  assignment  to  his  permanent  unit.  Contained  within  these 
programs  were  included  some  aspects  on  "leadership"  training  as  well  as 
"discipline".  For  the  enlisted  personnel  the  emphasis  was  on  discipline, 
the  necessity  of  promptly  obeying  lawful  orders  with  adherence  to 
organizational  norms  in  the  form  of  Naval  regulations.  For  the  officer 
the  emphasis  was  on  the  basic  responsibilities  as  a  leader  and  the  rudi¬ 
mentary  skills  needed  to  direct  the  efforts  of  others.  Leadership  con¬ 
tinued  to  remain  a  near  sacred  term,  steeped  in  tradition  and  assumed  to 
be  something  a  person  was  born  with  or  without.  Until  the  1950's, 
leadership  training  within  the  Navy  was  not  viewed  with  any  serious 
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concern  for  modification.  New  management  ideas  developed  by  the  civilian 
sector  were  largely  ignored  and  not  incorporated  within  any  Naval 
training.  In  a  Navy  where  disciplinary  standards  were  well  established; 
manpower  supplies  were  adequate  to  sustain  manning  levels;  and  training 
costs  for  necessary  skills  were  not  out  of  control,  there  was  no  perceiv¬ 
ed  need  for  further  leadership  training. 

B.  DEVELOPMENT  OF  NAVY  LEADERSHIP  TRAINING 
1.  The  1950’s 

With  the  introduction  of  the  Uniform  Code  of  Military  Justice 
[UCMJ]  into  the  Navy  in  1951,  the  nation's  concern  for  the  individual 
rights  of  soldiers  and  sailors  became  evident.  This  necessitated  an 
examination  of  the  traditional  leadership  style  then  practiced  throughout 
the  Navy.  Soon  this  concern  became  focused  on  symptoms  that  were  viewed 
as  evidence  of  poor  or  misdirected  leadership.  Sources  for  this  concern 
included:  responses  from  two  thirds  of  a  sample  of  10,000  U.  S.  Navy 
sailors  who  perceived  that  their  officers  and  petty  officers  were  not 
interested  in  them  as  human  beings;  a  brig  population  that  had  grown 
equal  in  size  to  the  entire  U.  S.  submarine  force;  and,  the  shocking 
information  that  some  70  percent  of  the  U.  S.  prisoners  of  war  in  Korea 
had  actively  collaborated  with  their  captors. [HUBBELL, 1960] 

Under  this  pressure  and  influence,  line  officers  along  with 
civilian  leaders  began  to  take  a  closer  look  at  leadership  training  with¬ 
in  the  Navy.  On  May  17,  1958,  General  Order  21  was  issued  by  the 
Secretary  of  the  Navy  to  all  commanding  officers  to  integrate  leadership 
training  into  the  technical  training  of  their  men. [CNP, 1963]  The  result 

of  this  order  was  a  succession  of  haphazard  training  efforts  that 
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lacked  any  clear  purpose,  theoretical  basis,  method  or  goal.  Individual 
training  programs  reflected  each  commanding  officer's  own  ideas,  educa¬ 
tional  background  and  knowledge  of  leadership,  as  well  as  how  to  best 
teach  it.  Regardless  of  the  ineffectiveness  of  General  Order  21,  it  did 
demonstrate  a  commitment  to  leadership  training  from  the  upper  echelon  of 
the  Navy.  However,  changing  emphasis  in  programs  reflected  a  continuing 
frustration  at  not  achieving  the  hoped  for  levels  of  productivity,  con¬ 
formity  and  retention  from  its  personnel. 

2.  The  1960's 

General  Order  21  did  not  meet  with  its  intended  success.  The 
initial  acclaim  and  status  that  the  leadership  training  program  gained  as 
a  separate  division  in  the  Bureau  of  Naval  Personnel  was  lost,  as  hopes 
faded  with  a  lack  of  tangible  results.  Reissuance  of  the  order  in  1963 
did  nothing  to  alleviate  the  situation.  In  1966,  Naval  leadership 
training  was  incorporated  into  the  broader  program  of  General  Military 
Training  [GMT]. [PARKER, 1980]  Leadership  training  requirements  in  GMT 
were  reduced  from  previous  levels.  Commanding  officers  were  committed  to 
train  their  enlisted  personnel  ten  hours  per  year  in  subjects  such  as 
leadership  style,  the  chain-of-command  and  authority,  responsibility  and 
accountabil ity.[AUEL,1975] 

Individual  commanding  officers,  still  responsible  for  implement¬ 
ing  the  training,  saw  little  evidence  of  clear  purpose  or  benefit  from 
the  program.  The  leadership  training  was  typically  delegated  to  a  less 
capable  or  already  overburdened  junior  officer.  The  decline  of  this 
leadership  program  marked  the  last  time  operational  commanding  officers 
would  be  entrusted  with  any  formal  leadership  training  program. 
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Succeeding  programs  would  be  centrally  controlled  and  conducted  by  the 
Navy  training  establishment.  The  decline  and  replacement  of  this  program 
typifies  the  course  of  such  efforts  within  the  Navy.  As  one  analysis 
suggests,  "The  leadership  program  fell  victim  to  its  own  frills  and  was 
downgraded  by  Navy  Institutionalists,  because  it  was  an  [Secretary  of  the 
Navy]  intervention  without  sufficient  input  from  line  managers." 
[AUEL, 1975] 

3.  The  1970*  s 

The  Navy  in  1970  was  in  a  state  of  rapid  change.  Society's 

problems  had  spilled  over  into  the  Navy  while  the  Vietnam  conflict  had 

severely  tarnished  the  military  image.  The  traditional  character  and 

structure  of  the  Navy  was  coming  under  question.  Drug  and  alcohol  abuse 

as  well  as  racial  problems  were  spreading.  Imminent  conversion  to  an 

all  volunteer  force  promised  only  additional  problems  for  Navy  leaders. 

Amid  this  social  upheaval  and  pressure  for  change,  a  relatively  young  and 

unconventional  Chief  of  Naval  Operations  [CNO],  ADM  Elmo  R.  Zumwalt,  Jr. 

assumed  command.  In  his  first  year,  only  nine  and  one-half  percent  of 

eligible  personnel  re-enlisted,  far  below  the  nimber  required  to  maintain 

mandated  personnel  strength  levels. [PARKER, 1980]  The  new  recruits 

necessary  to  reduce  this  shortage  would  ensure  that  societal  problems 

would  continue  to  infiltrate  and  dominate  the  military  scene.  ADM 

Zumwalt  expressed  his  leadership  philosophy  in  the  following  terms: 

...The  style  of  leadership  that  accorded  best  with  my 
own  inclinations  and  operational  efficiency  was  one  of 
treating  subordinates  with  consideration  and  respect.  I 
had  not  found  that  a  'tight'  ship  had  to  be  an  'uptight' 
ship  and  I  had  hoped  that  sooner  or  later  the  Navy  would 
give  institutional  recognition  to  this  principle. .. .by 
overhauling  such  of  its  procedures  and  jettisoning  such 
of  its  traditions  as  encouraged  martinetism  and 
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martinets . I  am  certain  that  what  finally  decided 

Secretary  of  Defense,  Melvin  Laird,  and  the  Secretary  of 
the  Navy,  John  Chaffe,  to  risk  junping  me  into  the  posi. 
tion  of  Chief  of  Naval  Operations  over  the  heads  of 
thirty.three  of  my  seniors  was  my  advocacy  of  rapid  and 
drastic  changes  in  the  way  the  Navy  treated  its  uniformed 
men  and  women. [ZUMWALT, 1976, p. 167.169] 

Using  a  group  of  hand  picked  senior  officers  in  the  Bureau  of 
Naval  Personnel,  ADM  Zumwalt  tried  to  establish  new  philosophies  and 
procedures.  Intending  to  bring  dignity,  pride  and  self-esteem  to 
sailors  and  to  end  the  "Mickey  Mouse"  rules,  ADM  Zumwalt  commenced  re¬ 
formulating  Navy  personnel  management  policies  with  his  infamous  "Z. 
Grams".l  Training  programs  in  the  areas  of  drug  and  alcohol  abuse, 
race  relations,  overseas  diplomacy  and  leadership  were  established.  The 
entire  effort  was  labeled  as  the  Human  Goals  Plan. 

One  of  the  most  controversial  and  opposed  aspects  of  the  Human 
Goals  Plan  was  its  top-down,  OMBUDSMEN- like  feature.  For  example.  Head, 
quarter's  "...program  managers  had  wide  latitude  to  intervene  at  any 
level  in  the  Navy  organization  with  stringent  requirements  for  individ¬ 
ual  ship  participation. . ."[AUEL, 1975]  This  feature  was  deeply  resented 
by  many  senior  Navy  officers  and  petty  officers.  They  claimed  Human 
Goals  mandated  actions  eroded  discipline,  took  too  long,  ignored 
operating  requirements,  and  resulted  in  a  loss  of  the  immediate  superi¬ 
or's  credibil ity. [AUEL, 1975]  Thus,  ADM  Zumwalt’ s  view  of  an  urgently 
needed  effort  to  rid  the  Navy  of  rigidity,  conflicted  with  his  oppo¬ 
nent's  views  of  a  breakdown  in  discipline  in  working  outside  of  the 
chain  of  command. 

IZ-Gram  was  a  phrase  used  to  denote  ADM  Zumwalt' s  CNO  policy 
memorandums.  Sent  simultaneously  to  all  levels  of  command,  this  rapid 
distribution  resembled  the  sending  of  a  telegram,  hence  the  phrase, 
Z-Gram. 
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Opposition  to  A0M  Zumwalt's  approach  and  its  unworkability  re¬ 
sulted  in  a  structural  change  in  late  1972.  The  program  continued,  with 
an  effort  to  return  it  back  under  the  chain-of-command.  "The  entire 
effort  had  grown  quickly,  contained  many  unqualified  people  in  important 
jobs  and  lacked  the  necessary  objectives  and  assessment  machinery  to  de¬ 
monstrate  what,  if  anything,  was  being  accomplished. "[PARKER, 1980, p. 7] 
However,  responsibility  and  reporting  arrangements  were  not  the  only 
problems.  “The  vague  goals  of  bringing  equity,  developing  management 
skills  and  improving  personnel  performance  were  well -intended,  but  un¬ 
coordinated,  unevenly  managed,  and  generally  ineffective. "[PARKER, 1980, 
p.7]  The  problems  were  compounded  by  a  lack  of  evaluation  to  de¬ 
termine  its  progress  or  overall  success. 

The  new  program.  Human  Resources  Management  [HRM],  represented 
more  of  a  change  in  name  and  structure  rather  than  content.  Fleet 
commanders  and  other  line  managers  were  delegated  the  responsibility  of 
continuing  these  programs.  The  leadership  training  program  continued 
basically  unaltered  with  the  exception  of  a  new  name.  Leadership  and 
Management  Training  [LMT].  LMT  consisted  of  a  ten  day  course  attended 
by  personnel  in  the  top  four  enlisted  pay  grades  and  the  lower  four 
officer  pay  grades.  Like  those  programs  that  preceeded  it,  LMT  lacked 
clear  objectives,  theoretical  basis,  and  a  plan  for  evaluating  its 
effectiveness. [PARKER, 1980]  In  addition,  each  school  developed  its 
own  curriculum  and  style  resulting  in  wide  differences  between  schools 
as  to  what  was  actually  being  presented. 
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Racial  incidents  in  1972  onboard  the  aircraft  carriers  Kitty 
Hawk  and  Constellation,  and  the  fleet  oiler  Hassayampa,  created  opposi¬ 
tion  to  ADM  Zumwalt's  programs  from  outside  the  Navy.  Many  placed  the 
blame  on  "permissiveness"  resulting  from  ADM  Zumwalt's  humanistic 
policies. [PARKER, 1980]  A  special  subcommittee  of  the  House  Armed 
Services  Committee  headed  by  Representative  Floyd  V.  Hicks,  in  calling 
for  greater  emphasis  on  leadership  training  in  the  Navy,  released  the 
following  statement: 

"One  of  the  most  alarming  features  of  the  investigations 
was  the  discovery  of  lack  of  leadership  by  middle  manage¬ 
ment  in  the  Navy.  It  became  apparent  that  while  junior 
officers,  chief  petty  officers  and  senior  petty  officers 
were  performing  their  technical  duties  in  a  proficient 
manner  there  was  a  lack  of  leadership  in  dealing  with 
seamen." [HOUSE  OF  REPRESENTATIVES, 1973] 

In  mid-1974  ADM  Zumwalt  was  replaced  as  CNO  with  ADM  James  R. 
Holloway,  III,  a  more  conventional  leader  in  terms  of  Navy  leadership. 
ADM  Holloway  was  concerned  with  the  seeming  lack  of  discipline,  lost 
pride,  and  absence  of  leadership  especially  within  the  ranks  of  the 
Navy's  middle  managers.  As  CNO,  he  set  as  one  of  his  primary  goals,  the 
improvement  of  senior  petty  officer  and  junior  officer  leadership  per¬ 
formance.  As  a  result,  on  28  August  1974  a  study  was  undertaken  to 
determine  the  cause  of  the  lack  of  leadership.  More  importantly,  the 
study  would  assess  the  needs  of  the  Navy  in  terms  of  leadership  training 
and  design  improvements  necessary  to  upgrade  LMT  to  meet  these  needs. 
This  action  would  ultimatly  lead  to  the  development  of  the  Navy's 
Leadership  and  Management  Education  and  Training  Program  [LMET],  but 

would  not  actually  commence  for  another  four  years. 
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C.  LEADERSHIP  AND  MANAGEMENT  EDUCATION  AND  TRAINING  [LMET] 

In  early  1975  the  Chief  of  Naval  Operations  [CNO],  ADM  James  L. 
Holloway,  III,  and  top  Navy  leaders  found  themselves  in  a  dilemma.  A 
comprehensive  review  and  examination  of  existing  leadership  and  manage¬ 
ment  training  had  revealed  58  formal  training  courses  and  11  correspond¬ 
ence  courses  costing  12.8  million  dollars  a  year. [AUEL, 1975]  Neverthe¬ 
less,  contemporary  Navy  problems  such  as  high  attrition,  low  retention, 
absenteeism,  substance  abuse  and  crisis  management  among  its  personnel 
were  continuing  to  be  sited  as  major  problems.  In  an  effort  to  rectify 
the  situation,  after  internally  generated  Navy  studies  had  produced  no 
conclusive  results,  several  civilian  contractors  were  asked  to  submit 
training  proposals  and  McBer  and  Company,  a  Boston-based  research  and 
development  firm,  was  chosen  to  develop  a  program  to  address  the  Navy's 
leadership  and  disipline  problems.  The  Navy  continued  with  its  broad 
based  leadership  training  [LMT],  with  emphasis  on  general  knowledge  of 
management  concepts  and  theories  until  McBer  cou^d  develop  the  proposed 
skill  acquisition  program  of  job  relevant,  specific  practices  and 
techni ques . 

1 .  The  Navy  LMET  Plan 

The  Navy's  plan  under  McBer  would  be  to  develop  Naval  personnel 
at  each  level  of  competence  required  by  their  jobs.  In  doing  so,  the 
new  program's  mission  would  be  "to  Increase  the  Navy's  ability  to 
achieve  its  overall  mission  by  increasing  the  effectiveness  of  Navy 
leadership  across  all  levels  of  the  chain-of-command."[CNET,N.D.,p.5] 
The  endeavor  would  be  a  systematic  training  program,  researched  based 
upon  Navy  situations  aimed  at  specific  billet  levels,  emphasizing 
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individual  initiative  and  accountability.  Specifically,  its  goals  would 
be: 


To  provide  a  formal  and  systematic  training  program  for 
professional  development  of  Navy  leaders  at  critical 
points  in  their  careers,  based  on  research  of  effective 
Navy  leadership. 

To  train  officers  and  petty  officers  in  the  specific 
leadership  and  management  skills  needed  to  perform 
effectively  at  their  level  in  the  chain-of -command. 

To  conduct  ongoing  evaluation  for  improving  and  updating 
these  programs. 

To  encourage  Navy  leaders  to  take  personal  responsibility 
for  implementing  effective  leadership  skills,  by  means  of 
an  educational  approach  that  emphasizes  individual  initi¬ 
ative  and  accountability  for  effective  performance  as  a 
Navy  leader.  [HRMC,N.0.,p.7] 

The  formal  Navy  training  plan  was  issued  in  February  1979, 

several  months  after  the  commencement  of  LMET  course  training.  That 

plan  addressed  the  results  expected: 

Improved  leadership  and  management  competence  on  the 
part  of  the  Navy's  officers,  petty  officers  and  civilian 
personnel  will  enhance  the  Navy's  performance  in  all 
mission  areas,  and  may  well  provide  the  margin  of  supe¬ 
riority  at  sea  that  the  Navy  can  achieve  over  any 
potential  future  adversary.  It  will  also  aid  in  the 
resolution  of  contemporary  Navy  problems  involving 
retention,  crisis  management,  disciplinary  rates, 
attrition,  working  conditions,  etc.  LMET  is  designed  to 
improve  and  maintain  the  requisite  level  of  leadership 
and  management  competence  through  the  Navy  total  force. 
[CN0.1979] 

2.  Theoretical  Bases  for  LMET 


Leadership  and  Management  Education  and  Training  [LMET],  the 
Navy's  sole  integrated  approach  to  the  leadership  training  problem  was 
developed  by  McBer  and  Company.  McBer  and  Company  was  set  up  in  1970  by 
Harvard  professor  David  C.  McClelland  and  psychologist  David  Berlew  - 
hence  the  name  McBer. 
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McC lei  land, an  eminent  clinical  psychologist,  is  well  known  for 
his  research  on  power  and  achievement  motives.[G0LEMAN,1981,p.35]  In 
his  book.  The  Achieving  Society,  he  discusses  research  which  sought  to 
isolate  certain  psychological  factors,  particulary  the  need  for  achieve¬ 
ment  and  to  assess  their  impact  upon  the  economic  growth  and  decline  of 
societies.  Using  scientific,  quantitative  methods,  coupled  with  the 
psychological  knowledge  of  human  motivation,  he  demonstrated  how  the  need 
to  achieve  motive  influenced  economic  development,  which,  more  important¬ 
ly,  could  better  equip  man  to  shape  his  own  destiny. [MCCLELLAND, 1961] 

Convinced  of  the  achievement  motive,  McClelland  undertook  re¬ 
search  in  the  area  of  motive  acquisition.  Both  behavior  theory  and 
psychoanalysis  agree  that  stable  personality  characteristics,  like 
motives,  are  laid  down  in  childhood  and  difficult  to  change. 
McClelland's  attempt  to  somehow  change  these  motives  in  adults  raised 
problems  of  both  an  ethical  as  well  as  a  methodological  nature.  Clearly 
there  were  processes  in  the  past  which  had  successfully  altered 
personality  change  -  most  notably  devout  missionaries .[MCCLELLAND, 1965, 
p.322]  Using  empirical  information  from  behavioral  learning  experi¬ 
ments,  a  strategy  was  set  up  in  an  attempt  to  change  or  promote  the 
achievement  motive,  thereby  improving  entrepreneurial  performance  of 
businessmen. 

McClelland's  motive  development  program  was  centered  around 
twelve  propositions  of  motive  acquisition,  notions  mostly  backed  by 
empirical  information  or  research  from  various  studies.  The  training 
program  was  designed  for  small  groups  [under  25]  to  be  taken  voluntarily 
over  a  short  duration  of  time  [1-3  weeks].  The  educational  design  of  the 
course  used  various  methods  of  content  and  process  presentations  to 
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achieve  the  overall  goal.  The  course  normally  ended  with  each  partici¬ 
pant  preparing  a  written  document  outlining  his  goals  and  life  plans  for 
the  next  two  years.  McClelland  states,  “The  participants  were  to  regard 
themselves  as  "in  training"  for  those  next  two  years,  in  that  10-14  days 
is  obviously  too  short  a  time  to  do  more  than  conceive  a  new  way  of  life: 
it  represents  the  residential  portion  of  the  training  only."  During 
these  two  years,  questionnaires  would  be  sent  out  each  6  months  to  remind 
them  of  the  issues  discussed  and  to  give  them  information  to  help 
determine  how  well  they  were  progressing  on  achieving  their  self  made 
goals. [McClelland, 1965, p.329] 

After  repeatedly  giving  the  course  both  at  home  and  abroad, 
McClelland's  data  seemed  to  suggest  that  one  third  of  the  people  remained 
relatively  unaffected  while  the  other  two  thirds  represented  a  doubling 
of  the  normal  rate  of  unusual  entrepreneurial  activity. [MCCLELLAND, 1965] 

McClelland's  propositions  were  stated  generally  enough  so  that 
other  terms  such  as  "attitude"  or  “personal ity  characteristics"  could  be 
substituted  for  the  term  "motive".  In  this  way  he  believed  the  proposi¬ 
tions  would  also  hold  true  for  other  personality  variables. [MCCLELLAND, 
1965, p .3323 

As  a  final  note  on  his  motive  development  program  of  training 
McClelland  wrote,  "...rather  than  developing  “all  purpose"  treatments, 
good  for  any  person  and  any  purpose,  it  [psychotherapy]  should  aim  to 
develop  specific  treatments  or  educational  programs  built  on  laboriously 
accumulated  detailed  knowledge  of  the  characteristic  to  be  changed." 
[MCCLELLAND, 1965, p.333]  McClelland's  motive  development  program  was  to 
be  a  significant  building  block  in  what  would  become  the  Navy’s  LMET 
Program. 


27 


In  early  1971,  McClelland  [MCCLELLAND, 1973],  criticized  the 
intelligence  or  aptitude  testing  movement  in  a  public  lecture  given  at 
the  Educational  Testing  Service  in  Princeton,  New  Jersey.  McClelland 
criticized  the  intelligence  tests  just  as  he  had  done  15  years  earlier, 
on  the  grounds  that  while  they  may  predict  good  grades  in  school  they 
have  little  effect  in  predicting  good  life  outcomes.  His  own  research 
supports  this  notion.  Good  test  scores  allow  people  to  get  into  better 
schools,  however,  after  graduation  the  significance  ends  -  good  test 
score  and  bad  test  score  graduates  both  succeed  and  fail.  McClelland 
argues  that  a  much  wider  array  of  talents  or  compentencies  should  be 
assessed  to  determine  college  entrance,  with  an  emphasis  on  "grades  in 
life"  and  not  "grades  in  school" .[MCCLELLAND, 1973, p. 7]  Good  grades  in 
school  falsely  lead  people  to  believe  that  they  are  more  competent  and 
therefore  more  likely  to  do  well  in  life. 

McClelland  listed  six  principles  as  paramount  in  his  alternative 
approach,  compentency  testing,  to  traditional  intelligence  testing.  Of 
the  six,  two  are  particularly  significant  in  understanding  overall  LMET 
development. 

The  first  principle  concerns  criterion  sampling.  Criterion 
sampling  means  observing  people  in  the  field  and  analyzing  their  perfor. 
mance.  A  test  that  is  directed  at  the  components  of  this  performance 
would  be  the  best  test.  McClelland  states,  "If  you  want  to  test  who  will 
be  a  good  policeman,  go  find  out  what  a  policeman  does.  Follow  him 
around,  make  a  list  of  his  activities,  and  sample  from  that  list  in 
screening  appl icants ."[MCCLELLAND, 1973, p. 7]  General  intelligence  and 

the  ability  to  play  word  games  may  not  be  the  best  test  in  determining 


who  to  hire  as  a  policeman  -  a  movement  toward  behavioral  analysis  is 
required. 

The  second  finciple  concerns  the  ability  to  assess  competencies 
involved  in  clusters  of  life  outcomes.  While  criterion  sampling  can  be 
used  to  identify  occupational  competencies  unique  to  a  special  job  or 
task  it  is  also  necessary  to  identify  social  competencies  that  contribute 
to  successful  performance.  McClellano  writes  that,  "Some  of  these 
competencies  may  be  traditional  cognitive  ones  involving  reading  and 
writing.  Others  should  involve  what  traditionally  have  been  called 
personality  variables,  although  they  might  better  be  considered 
competencies. "[MCCLELLAND, 1973, p. 10]  McClelland  gives  four  illustra- 

tions  of  these  social  competencies,  each  with  supportive  research  for 
both  its  significance  as  well  as  insights  into  its  measurement,  they  are; 
communication  skills,  patience,  moderate  goal  setting,  and  ego 
development. [MCCLELLAND, 1973] 

3.  LMET  Study  for  the  Navy 

Using  McClelland's  research  and  conceptual  ideas  on  job  compe¬ 
tency,  McBer  and  Company,  working  in  close  cooperation  with  the  Navy's 
Bureau  of  Personnel  [8UPERS],  began  developing  LMET  in  1976.  Similar 
methods  and  techniques  used  during  the  past  decade  in  civilian  organi¬ 
zations  were  adapted  for  the  Navy's  overall  LMET  design.  The  first  task 
was  to  identify  what  Navy  leaders,  particularly  superior  Navy  leaders 
actually  do  in  handling  leadership  and  management  tasks.  As  Dr.  D.  G. 
Winter  explains: 

The  traditional  way  of  answering  the  question,  "What 
makes  a  good  Navy  leader?",  is  to  ask  people  what 
personal  qualities  and  skills  they  believe  are  important 
for  leadership  and  management  in  a  particular  job. 


Putting  together  the  answers  of  people  actually  In  the 
job,  the  answers  of  superiors  [or  supervisors],  and  the 
answers  of  their  subordinates  [or  clients]  ought  to  give 
us  a  collective  theory  of  what  that  job  requires.  In 
fact,  however,  this  procedure  is  likely  to  give  us  a 
mixture  of  highly  specific  job  tasks  and  vague  plati. 
tudes  about  personal  qualities,  while  neglecting  the 
qualities  that  are  crucial  for  excellent  or  superior 
performance.[WINTER,1979,p.2] 

McBer  believes  that  it  is  more  accurate  and  relevant  to  find 
out  who  are  the  superior  and  average  leaders  from  the  personnel  working 
closest  with  them,  and  then  identify  what  distinguishes  them  from  each 
other. 

Using  Navy  commands  in  San  Diego,  California  and  Norfolk, 
Virginia,  McBer  asked  commanding  officers  to  identify  superior  and 
average  leaders  at  eight  career  points:  division  officer,  department 
head,  executive  officer,  and  commanding  officer  for  commissioned 
officers,  and  petty  officer,  leading  petty  officer,  leading  chief  petty 
officer,  and  master  chief  petty  officer  of  the  command  for  non.commis. 
sioned  officers.  In  this  way,  51  people  [30  superior,  21  average]  were 
identifed  from  the  Pacific  Fleet  and  78  people  [38  superior,  40  average] 
from  the  Atlantic  Fleet.  Most  combinations  of  warfare  community  and 
career  points  were  represented  in  this  total  sample  of  129  officers  and 
enlisted. [WINTER, 1979] 

The  personnel  were  interviewed  individually  and  asked  to  relate 
three  incidents  in  which  they  /el  t  they  did  very  well,  and  then  three 
incidents  in  which  they  did  not  feel  very  successful  in  their  present 
jobs. 2  The  interviews  were  recorded  almost  verbatim  and  analyzed 

?The  McBer  approach  is  not  unique  in  all  respects.  The 
critical  -  incident  interview  was  developed  by  psychologist  John 
Flanagan  during  WW  I I . ]G0LEMAN, 1981 ,p . 39]  McBer  calls  his  data 
gathering  technique  behavioral  event  interviews  [BEI]. 
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carefully  to  determine  what  similarities  superior  Naval  leaders  had  shown 
that  average  leaders  failed  to  show.  The  interviews  of  36  of  the  Pacific 
Fleet  sample  were  used  to  create  a  series  of  27  competency  elements, 
grouped  into  5  clusters.  When  scored  on  the  whole  sample  of  51  Pacific 
Fleet  interviews,  most  of  the  27  competency  elements  differentiated  the  2 
groups. [WINTER, 1979] 

To  validate  their  findings,  McBer  first  used  the  Atlantic  Fleet 
sample  scored  by  interviewers  who  did  not  know  if  the  person  was 
"superior"  or  "average".  Additionally,  a  second  validation  technique  was 
used  involving  a  new,  much  larger  sample.  An  extensive  battery  of  paper 
and  pencil  tests  was  developed  and  administered  to  over  1,000  Navy 
personnel  from  petty  officer  through  commanding  officer,  in  the  3  warfare 
communities  from  both  fleets.  Interviews  which  could  be  scored  for 

competency  elements  were  available  on  61  of  these  persons .[WINTER, 1979] 
Further  analysis  was  conducted  using  220  persons  from  the  larger  sample 
size  predominately  in  determining  variations  of  leadership  and  management 
skills  on  a  hierarchial  level. 

Once  a  competency  element  derived  from  the  original  interviews 
with  the  sample  of  Pacific  Fleet  personnel  had  been  validated  by  either 
of  those  two  procedures,  it  was  considered  to  be  a  competency  that  is 
associated  with  superior  leadership  and  management  performance  in  the 
Navy.  Sixteen  of  the  original  27  elements  were  validated  in  this  way. 3 
[WINTER, 1979]  Those  27  initially  identified  elements  served  as  the  basis 
for  original  LMET  curriculun  design. [PARKER, 1980, p. 15] 

^For  a  more  indepth  description  of  competency  identification  and 
validation  the  reader  should  consult  the  Winter  Report  [WINTER, 1979]  on 
LMET  theory  and  research. 
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These  16  validated  competencies  are  at  the  core  of  the  LMET 
Program.  The  basic  premise  of  LMET  is  that  increases  in  any  or  all  of 
those  competencies  will  be  associated  with  improved  leadership 
performance  in  the  Navy. [WINTER, 1979]  Today,  the  27  competency 
elements  have  subsequently  been  reduced  to  16  competencies  which  are 
grouped  into  5  competency  clusters  for  training  purposes. * 

4.  The  LMET  Competencies 

A  competency  is  any  knowledge,  skill,  attitude  or  value  which 
can  be  shown  to  distinguish  reliably  between  effective  and  less 
effective  job  performance.  As  Richard  Boyatzis,  President  of  McBer  and 
Company  explains,  "Competencies  are  not  aspects  of  a  job,  but  rather 
characteristics  of  the  people  who  do  the  job  best. "[GOLEMAN, 1981 ,p.40] 
In  other  words,  a  competency  is  what  superior  performers  do  more  often, 
in  more  situations,  for  better  results  than  do  average  performers. 

The  first  competency  cluster  is  a  concern  for  efficiency  and 
effectiveness,  paraphrased  as  "doing  things  well,  and  wanting  to  do 
better. "[HRMC, N. D, p. 1 1A]  Major  components  of  this  group  include 
setting  goals  and  performance  standards,  and  taking  initiative,  with 
the  concern  for  achievement  as  an  underlying  thought. 

The  second  competency  cluster  is  the  skillful  use  of  influence 
or  "using  influence  in  a  positive  fashion. . .not  as  a  personal  end  «  but 
toward  Navy  goals  and  effectiveness. "[HRMC, N.D.p.l2A]  Major  components 
include  Influences,  team  building,  development  of  subordinates  and 
self-control,  with  the  concern  for  influence  as  an  underlying  thought. 


*A  complete  list  of  the  5  competency  clusters  is  contained  in 
Table  1. 
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TABLE  1 

LMET  COMPETENCY  CLUSTERS 


1.  Efficiency  and  Effectiveness  -  “Doing  things  well,  and  wanting  to  do 

better."  Components  include: 

*  Setting  goals  and  performance  standards 

*  Taking  initiative 

2.  Skillful  Use  of  Influence  -  "Using  influence  in  a  positive  fashion,.. 

not  as  a  personal  end  but  toward  Navy  goals  and  effectiveness..." 
Components  include: 

*  Influences 

*  Team  builds 

*  Develops  subordinates  [coaches] 

*  Self, control 

3.  Advising  and  Counseling  .  "Advise  and  counsel  personnel  in  order  to 

improve  their  performance  on  the  job."  Components  include: 

*  Positive  expectations 

*  Realistic  expectations 

*  Understanding 

4.  Management  Control  -  "Optimizing  people  and  resources  to  the  task." 

Components  include: 

*  Plans  and  organizes 

*  Optimizes  use  of  resources 

*  Delegates 

*  Moniters  results 

*  Rewards 

*  Disciplines 

5.  Conceptual  Thinking  ,  "Applies  concepts  to  a  job  situation." 

Components  include: 

*  Conceptualizes 


Reference:  U.  S.  Navy  Human  Resource  Management  Center,  LMET 
Overview  Brief,  no  date. 


The  third  competency  cluster  is  that  of  advising  and 
counseling,  further  amplified  as  to  "advise  and  counsel  personnel  in 
order  to  improve  their  performance  on  the  job."  [HRMC.N.D. ,13A]  Major 
components  of  this  group  include  positive  expectations,  realistic  ex¬ 
pectations  and  understanding. 

The  fourth  competency  cluster  is  that  of  management  control  or 
to  "optimize  people  and  resources  to  the  task."  [HRMC.N.D. ,14A]  Major 
compontents  are  plans  and  organizes,  optimizes  use  of  resources, 
delegates,  monitors  results,  rewards  and  disciplines. 

The  final  competency  cluster  is  that  of  conceptual  thinking  or 
"identifying  and  organizing  relevant  facts  to  gain  a  clear  understand¬ 
ing  of  the  situation  before  acting  on  it."  [HRMC.N.D. ,15]  The  only 
component  of  this  cluster  is  that  of  applying  concepts  to  a  job 
situation. 

In  addition  to  the  five  competency  clusters,  the  McBer  study 
identified  six  basic  categories  of  leadership  styles  used  by  the 
officers  and  petty  officers  interviewed,  they  are: 

a.  Coercer  -  all  stick  and  no  carrot 

b.  Authoritarian  -  firm  but  fair 

c.  Affiliator  -  people  first,  task  second 

d.  Democrat  -  participative  manager 

e.  Pace  Setter  -  do  it  myself 

f.  Coach  -  Management  by  Objectives  [MBO]  Manager 

McBer  found  that  superior  leaders  tend  to  be  more  skilled  in  several 
styles,  recognizing  which  style  fits  a  given  situation.  LMET  en¬ 
courages  situational  leadership  with  the  development  of  a  larger  re¬ 
pertoire  of  leadership  styles. [HRMC,N.D.,p. 16] 
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5.  LMET  Structure 

LMET  Is  now  the  formally  approved  model  for  Navy  leadership 
training,  although  it  is  not  yet  completely  implemented.  To  achieve 
standardized  implementation  of  the  program  throughout  the  Navy,  the 
Chief  of  Naval  Education  and  Training  [CNET],  has  been  designated 
training  agent. 

The  first  LMET  classes  opened  in  1978,  and  the  program  has 
expanded  rapidly.  Projected  yearly  outputs  for  fiscal  years  80-84  are 
4,840  officers  and  12,242  enlisted  based  upon  the  nunber  of  school 
quotas  available. [HRMC.N.D.]  There  are  currently  five  levels  of  LMET 
with  special  emphasis  on  fleet  personnel  attending  first.  These  levels 
include;  commanding  officer/executive  officer,  department  head, 
division  officer,  chief  petty  officer  and  petty  officer.  Officer  LMET 
is  further  subdivided  with  separate  classes  for  aviation,  submarine, 
and  surface  ship  officers.[HRMC,N.D. ]  Personnel  returning  to  the 
fleet  normally  receive  LMET  enroute  to  their  new  permanent  duty 
station.  School  quotas  can  also  be  obtained  by  Navy  commands  for 
personnel  already  at  their  command  on  a  temporary  duty  basis.  However, 
these  quotas  are  extremely  difficult  to  obtain  and  are  offered  only  on 
a  space  available  basis. 

LMET  is  now  taught  at  existing  Navy  training  centers.  Sites  on 
the  east  coast  include:  Memphis,  Tennessee;  Little  Creek,  Virginia;  New 
London,  Connecticut;  Mayport,  Florida;  Pensacola,  Florida;  Newport, 
Rhode  Island;  and  Charleston,  South  Carolina.  Sites  on  the  west  coast 
are:  Coronado,  California;  San  Diego,  California;  Bangor,  Washington; 
Treasure  Island,  California;  and  Pearl  Harbor,  Hawaii. 
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6.  LMET  Instruction 


All  classes  last  10  working  days,  except  for  the  12  week  LMET 
instructor  course  taught  at  Memphis,  Tennessee.  Instructors  are  deemed 
to  be  the  key  to  success  of  the  program.  Accordingly,  instructors  have 
been  selected  against  rigorous  criteria  and  trained  in  LMET  concepts 
and  methods  by  the  program  designers.  No  one  is  allowed  to  teach  LMET 
who  is  not  a  graduate  of  the  12  week  LMET  instructor  course  .[0LMSTEA0, 
1980] 

LMET  class  sessions  are  team-taught  by  two  or  three  instructors, 
with  officers  teaching  officer  courses  and  senior  petty  officers  teach¬ 
ing  equivalent  enlisted  personnel.  Reportedly,  classes  generally  consist 
of  20-24  students. 

Each  LMET  course  is  based  around  the  competency  notion  and  the 
five  competency  clusters.  The  training  sequence  for  each  competency 
consists  of  lectures  to  identify  and  assess  how  the  competency  applies 
in  the  Navy.  Through  self-analysis  exercises  and  pretesting,  students 
discover  how  they  learn,  as  well  as  the  type  of  management  styles  they 
possess.  Skill  acquision  and  practice  are  developed  through  games, 
role-play  exercises  and  case  studies.  Finally,  through  various  pro¬ 
cesses  the  students  learn  to  apply  the  competencies  to  tasks  similar  to 
those  which  will  be  encountered  in  their  new  job  assignment. 

The  personal  comprehensive  plan  is  the  final  activity  in 
all  LMET  courses.  It  is  a  statement  of  personal  goals, 
shaped  to  the  concepts  and  language  of  the  leadership 
competency  model .  Participants  are  encouraged  to  formulate 
and  describe  realistic  yet  challenging  goals  that  are  appro¬ 
priate  to  their  Navy  leadership  situations,  to  become  aware 
of  difficulties  and  obstacles  to  these  goals,  and  to  nrlte 
out  specific  action  steps  to  overcome  the  obstacles  and 
attain  the  goals. 
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A  distinctive  feature  of  LMET  courses  is  the  personal  student 
log,  which  the  participant  retains  as  a  written  record  of  the 
LMET  experience.  It  contains  his  or  her  self-assessment,  life 
situation,  and  goals  -  all  phrased  in  terms  of  the  competency 
language. [WINTER, 1979, p.11-12] 

Differences  [PARKER, 1980]  between  classes  for  higher  or 
lower  ranking  personnel  are  found  primarily  in  the  content,  setting,  and 
cast  of  characters  used  in  the  role-play  situations,  case  studies  and 
general  class  discussions.  Chief  petty  officer  and  petty  officer 
courses  have  the  same  level  of  emphasis,  differing  primarily  in  the 
seniority  of  instructions  that  teach  the  two  levels. 

The  LMET  classes  are  reportedly  [PARKER, 1980]  well-paced, 
shifting  from  lecture  to  discussion  to  small  group  activities  in  a 
fashion  intended  to  maintain  student  interest,  to  provide  frequent 
opportunities  for  students  to  express  feelings  and  opinions  in  a 
supportive  environment,  and  to  trade  ideas  with  their  peers.  In  actual 
practice  LMET  courses  may  differ  slightly  from  location  to  location  and 
class  to  class  as  instructors  seek  to  motivate  and  influence  each 
particular  group.  Student  critiques  completed  at  the  end  of  the  course 
indicate  that  students  like  LMET. [PARKER, 1980] 

7.  Future  for  LMET 

As  mentioned  earlier,  LMET  has  not  yet  been  completely 
implemented.  Planning  through  fiscal  year  84  will  include  courses  for 
more  junior  petty  officers  as  well  as  shore  commands.  Navy  civilians, 
and  Naval  Reserve  personnel,  with  the  overall  goal  of  training  everyone 
in  a  Navy  leadership  position.  The  cost  of  LMET  during  1980  was  17 
million  dollars;  for  1982-86,  the  projected  cost  is  slightly  more  than 
29  million  dollars  per  year. [PARKER, 1980] 


Curriculun  modification,  concentrated  at  the  LMET  Instructor 
School  in  Memphis,  has  gone  on  almost  continously  since  the  problem 
began.  Efforts  to  improve  the  program  are  concentrated  in  Memphis  under 
the  direction  of  CNET.  Over  and  above  internal  course  improvement  and 
development,  however,  LMET  has  three  longer-range  goals: 

a.  Additional  research  is  planned  to  provide  further  vali¬ 
dation  and  cross-val idation  of  the  competencies  themselves.  Of 
particular  concern  here  is  pinpointing  in  greater  detail  the  re. 
lative  importance  [and  hence  the  appropriate  instructional 
weighting]  of  each  competency  at  a  few  "key"  Navy  billets  -  for 
example,  at  the  commanding  officer  level  and  at  the  petty 

officer  level. 

b.  The  test  battery  will  be  revised  and  expanded,  in  order 
to  provide  instruments  and  procedures  that  can  be  used  to  assess 
individual  standards  of  performance  against  the  competency 
profile  of  superior  Navy  leaders.  When  available,  such  instru¬ 
ments  could  also  be  used  before  and  after  courses,  as  measures 
of  the  immediate  effectiveness  of  a  person's  participation  in 
LMET. 


c.  The  LMET  program  will  be  evaluated.  Short-term,  inter¬ 
nal  course  evaluation  will  provide  an  immediate  indication  of 
the  effects  of  LMET.  Most  important,  however,  is  the  more 
difficult  long-term  external  evaluation.  Do  LMET  graduates 
perform  at  significantly  higher  levels  of  leasership  than  non¬ 
graduates?  Does  the  LMET  program  have  a  measurable  impact  on 
standard  Navy  indicators  of  leadership  and  management  perform¬ 
ance,  as  well  as  on  newer  indicators  that  will  be  developed  as  a 
part  of  LMET  itself?  [WINTER, 1979, p. 12-13] 


D.  NEED  FOR  LMET  EVALUATION 

LMET  will  soon  begin  its  fourth  year  with  graduates  continuing  to  be 
dispersed  throughout  the  entire  Navy  community,  limited  only  by  cost  con. 
serations  and  the  shear  nunbers  of  quotas  available  annually.  Plans  to 
continue  LMET  implementation  on  a  more  widespread  basis  are  ongoing  with 
the  full  backing  of  the  CNO.  The  implication  is  that  LMET  has  become  a 
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solution,  in  great  part,  to  the  Navy's  overall  leadership  and  discipline 
problem.  Most  people  feel  good  about  the  direction  of  LMET  and  its 
results,  however,  elaborations  are  few  and  comments  are  less  than 
specific. 

Just  how  good  is  LMET?  Is  LMET  producing  the  desired  results  or 
changes?  Unfortunately,  those  questions  remain  unanswered.  It  is  diffi¬ 
cult  to  identify  what  is  meant  by  "improving"  effectiveness.  Efficiency 
can  usually  be  measured  in  terms  of  cost  and  time.  Defining  effective¬ 
ness  requires  a  detailed  understanding  of  the  variables  that  affect 
performance. 

Several  prominent  industrial  psychologists  express  doubt 
about  the  prospect  of  meaningful  evaluations  of  methods  like 
McBer's.  It  is  simply  too  difficult  to  identify  the  qualities 
of  good  managers,  they  say,  let  alone  measure  their  validity  as 
predictors  of  performance.  [G0LEMAN,1981,p.46] 

Neither  McBer  nor  its  clients  have  so  far  produced  much  empirical 
proof  that  their  method  does,  in  fact,  lead  to  demonstrable  improvements 
in  job  performance.  There  are  few  published  studies  of  the  effectiveness 
of  comptenecy-based  selection  and  training  as  practiced  by  McBer. 
[G0LEMAN,1981] 

Much  of  McBer's  work  is  too  recent  for  meaningful  evaluation.  Ex¬ 
cept  for  end  of  course  critiques  by  students  and  informal  comments  by 
commanding  officers,  LMET  falls  into  this  category. 

However,  the  honeymoon  for  LMET  is  beginning  to  end.  Researchers 
are  starting  to  question  the  McBer  approach  as  unvalidated  theory,  ignor¬ 
ing  a  host  of  pet  factors,  such  as  situational  leadership  or  content 

t 

verses  process  leadership,  to  name  only  two.  Some  claim  the  data  gather¬ 
ing  and  analysis  are  not  particularly  impressive.  Others  insist  that 
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the  sample  size  to  base  this  magnitude  of  a  program  on  was  just  too 
small.  [PARKER, 1980] 

Additionally,  there  exists  a  very  real  need  for  all  levels  of  Navy 
managers  to  know  the  degree  to  which  the  LMET  Program  is  effective. 
Optimum  use  of  scarce  resources  is  a  major  concern  to  all  managers,  and 
the  LMET  Program  continues  to  make  demands  for  both  manpower  and  money 
without  proven  justification.  This  concern  should  question  whether  the 
ten  days  at  LMET  school  is  in  fact  improving  individual  effectiveness  as 
a  leader,  and  hence  worthwhile.  If  LMET  is  only  a  nice-to-know 
management  course,  producing  few  measurable  results,  then  there  would  be 
a  strong  argument  that  those  resources  could  be  better  allocated.  On  the 
other  hand,  if  LMET  graduates  are  more  effective  leaders  and  managers  in 
directing  the  manpower  resources  available  to  them,  then  the  program 
should  be  valued.  The  bottom  line  is  that  Navy  managers  have  a  real  need 
for  an  LMET  program  evaluation  to  determine  if  the  current  level  of 
support  is  warranted. 

An  evaluation  would  also  provide  LMET  program  managers  with  feedback 
as  to  its  effectiveness.  It  would  identify  areas  that  require  further 
emphasis  as  well  as  identify  deficiencies  if  they  exist.  The  fact  that 
continued  evaluation  of  the  program  is  a  stated  goal  [WINTER, 1979] 

further  highlights  this  need. 

Perhaps  more  importantly,  an  evaluation  can  provide  future  direction 
for  the  program.  This  aspect  can  keep  the  LMET  program  ahead  of,  or  at 
least,  on  top  of  emerging  problems,  thereby  assuring  it  remains  viable 
and  healthy.  With  this  forward  looking  attitude  the  program  can  become 
one  of  action  vice  reaction  to  the  needs  of  the  Navy. 
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The  list  continues  on  and  so  does  the  need  for  LMET  evaluation.  As 
with  any  program  in  times  of  austere  funding,  LMET  will  sooner  or  later 
come  under  careful  scrutiny  as  to  its  cost  effectiveness.  The  Naval 
Audit  Service  has  already  expressed  an  interest  in  examining  LMET  re¬ 
sults. [PARKER, 1980]  The  Navy  can  little  afford  not  to  evaluate  the 
program. 

As  discussed  at  length  above,  the  need  for  an  evaluation  of  the  LMET 
program  is  quite  clear.  Yet,  the  question  of  how  to  conduct  this  evalua¬ 
tion  remains.  How  to  best  evaluate  a  system-wide  program  as  extensive  as 
LMET  within  the  Navy  is  not  readily  apparent.  The  number  of  proposed 
methods  would  likely  equal  the  number  of  people  querried.  System-wide 
surveys,  periodic  graduate  questionnaries,  unit  reports  on  overall 
readiness  vs  number  of  graduates  onboard,  spot  interviews,  etc.,  are  all 
possible  approaches  to  evaluation  complete  with  strong  arguments  both  for 
and  against  their  use. 

In  deciding  on  the  issue  of  how  best  to  evaluate,  one  should 
consider  what  is  being  evaluated.  LMET,  as  well  as  any  program's 
effectiveness,  is  not  easily  measured.  There  are  few  clear  cut  effect¬ 
iveness  indicators,  no  impeccable  standards,  and  no  completely  reliable 
method  of  measuring  effectiveness  as  has  been  discussed  earlier. 

Another  concern  is  the  cost  of  any  evaluation  in  terms  of  time, 
manpower  and  money.  All  three  constraints  further  confuse  the  Issue  of 
how  to  best  evaluate  the  LMET  Program  while  optimizing  these  resources. 

This  research  is  intended  to  provide  program  managers  and  decision 
makers  with  a  practical  method  to  answer  the  very  question  of  how.  This 
method  of  initial  evaluation,  in  the  form  of  a  Pilot  Study  can  test 
hypotheses,  evaluate  a  methodology,  uncover  relevant  issues  while 
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providing  data  concerning  the  actual  evaluation  of  a  major  program.  This 
thesis  will  attempt  to  demonstrate  the  value  and  usefulness  of  the  Pilot 
Study  concept  as  related  to  a  preliminary  program  evaluation. 


It  is  under  this  context  that  this  research  on  IMET  effectiveness 
was  undertaken.  The  thrust  of  the  work  is  twofold,  first  to  try  to  shed 
some  light  on  LMET  effectiveness.  Second  and  possibly  more  important,  to 
demonstrate  the  practical  value  of  a  pilot  study  to  help  identify  the 
relevant  issues,  evaluate  methodology,  and  test  hypotheses  prior  to 
committing  extensive  resources  to  a  full  scale  evaluation. 
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III.  CONSIDERATIONS  IN  DESIGN  OF  AN  EVALUATION  OF  LMET : 
- GUIDELINES'  FOR  T  PTIOT  STUGY - 


A.  RATIONALE  FOR  A  PILOT  STUDY 

With  the  need  for  an  evaluation  firmly  established,  there  remains  at 
least  two  critical  questions  to  be  answered;  who  is  to  conduct  the  study 
and  how?  Let  us  briefly  examine  the  who  question  first.  There  is 
logical  support  for  McBer  and  Company  to  evaluate  the  LMET  Program,  since 
they  are  intimately  familiar  with  its  designed  structure  and  theoretical 
basis.  However,  there  exists  a  possible  conflict  of  interest  whereby  the 
credibility  of  results  could  be  in  jeopardy.  This  in  essence  would  be  a 
form  of  self-evaluation,  open  to  criticism  as  being  favorably  biased, 
regardless  how  fair  or  objective  it  was  in  reality. 

CNET  is  another  possible  choice  for  conducting  the  evaluation.  C NET 
has  familiarity  with  course  content,  teaching  techniques,  as  well  as  easy 
access  to  graduates  both  before  and  after  the  training.  Again  for  many 
similar  reasons,  the  outcome  of  such  an  evaluation  might  be  suspect.  As 
part  of  the  Navy  bureaucracy,  CNET  would  be  placed  in  an  untenable 
situation  of  evaluating  a  program  that  it  administers.  Any  such 
evaluation  would  probably  be  subject  to  close  scrutiny,  reguardless  of 
the  actual  quality.  Clearly,  there  is  a  need  for  an  independent 
evaluation  of  the  effectiveness  of  this  program  to  insure  a  relatively 
unbiased  outcome. 

This  leads  to  a  third  alternative,  that  of  an  independent  outside 
contractor  with  no  vested  Interest  in  the  outcome.  Yet  even  this  alter¬ 
native  has  considerable  drawbacks.  The  cost  of  an  evaluation  would  be 
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high,  especially  for  a  contractor  who  is  unfamiliar  with  the  program.  It 
would  be  necessary  for  the  contractor  to  expend  a  considerable  amount  of 
time,  manpower  and  money  in  studying  the  program  prior  to  commencing  the 
evaluation.  Even  after  that  initial  familiarization  phase,  there  exists 
a  very  real  possibility  that  the  program  could  be  misinterpreted, 
resulting  in  an  inappropriate  direction  for  data  collection.  This  could 
result  in  an  evaluation  that  uncovers  nothing  meaningful,  or  worse,  mis- 
evaluates  the  program,  causing  additional  expenditures  to  correct  non- 
existant  problems  as  well  as  creating  the  need  for  another  evaluation. 

The  need  for  an  evaluation  by  a  knowledgable  but  independent  third 
party  having  no  career-crucial  or  monetary  interest  in  the  outcome  points 
to  yet  another  alternate  approach.  An  independent  contractor  working 
with  only  a  sketchy  concept  of  the  program  and  little  inside  knowledge 
may  not  be  a  good  solution.  However,  if  that  same  contractor  were  pro¬ 
vided  with  some  meaningful  and  concise  direction  for  the  evaluation,  then 
the  study  could  be  conducted  more  efficiently  with  the  necessary  credi¬ 
bility.  One  method  to  provide  this  guidance  is  through  a  pilot  study. 
The  pilot  study,  conducted  by  personnel  knowledgable  of  the  organization 
and  the  program  being  evaluated,  could  identify  issues,  develop  hypo¬ 
theses,  and  even  test  an  evaluation  methodology.  By  conducting  a  pilot 
study  prior  to  the  full-scale  evaluation,  information  could  be  uncovered 
that  would  provide  the  program  manager  and/or  contractor  with  the  needed 
direction  and  relavent  Issues  to  effectively  conduct  a  valid  evaluation. 
Manpower  needs  for  such  a  pilot  study  are  minimal  with  a  variety  of 
possible  sources  such  as  CNET,  HRMC's  or,  as  in  this  case.  Naval  Post¬ 
graduate  students  working  on  their  master's  degree  thesis. 
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It  was  with  these  thoughts  in  mind  that  a  pilot  study  of  the  Navy's 
LMET  Program  was  undertaken  by  two  Naval  Postgraduate  students.  The 
study  commenced  in  mid-December  1980  and  was  completed  in  mid- June  1981, 
a  period  of  approximately  six  months. 

B.  EVALUATION  DESIGN  CONSIDERATIONS 

Having  decided  who  should  conduct  the  pilot  study,  the  remaining 
question  was  how  to  evaluate  the  program  and  from  there  design  the  pilot 
study.  The  ideal  design  would  have  been  to  establish  a  control  group  of 
non-LMET  graduates  and  compare  their  leadership/management  practices  with 
LMET  graduates  to  see  if  there  were  any  recognizable  differences.  Then, 
compare  the  non-LMET  group  with  the  LMET  group  to  determine  which  command 
was  in  fact,  best  managing  those  problem  areas  which  LMET  was  intended  to 
alleviate.  These  would  include  areas  such  as  drug  and  alcohol  abuse, 
retention,  crew  morale,  unit  readiness  and  material  condition,  and 
command  climate.  Such  an  approach  to  evaluating  a  program  would  be  very 
scientific  and  lend  itself  in  establishing  the  effectiveness  of  a  parti¬ 
cular  program.  The  problem  with  implementing  this  approach  is  twofold. 
First,  no  such  control  group  was  established  when  LMET  was  started.  As  a 
result  every  command  in  the  Navy  has  some  LMET  graduates  on  board. 
Secondly,  there  is  the  problem  of  determining  if  a  command's  management 
of  those  problem  areas  was  in  fact  due  to  LMET  training,  some  other 
factor,  or  more  likely,  a  combination  of  more  than  one  factor. 

Lack  of  a  control  group  does  not  mean  that  a  useful  evaluation  can 
not  be  accomplished.  It  does,  however,  lend  support  to  conducting  a 
pilot  study  in  order  to  test  an  evaluation  method  prior  to  committing 
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scarce  resources  to  a  full  scale  evaluation.  If  unable  to  compare  two 
groups  such  as,  LMET  and  non-LMET  graduates,  then  the  next  logical  step 
is  to  compare  individual  performance  before  and  after  LMET  training.  By 
collecting  behavioral  data  on  managers  before  LMET  school  and  comparing 
it  to  behavioral  data  gathered  sometime  after  LMET  school,  changes  in 
leadership/managerial  behavior  could  be  detected.  The  implication  is 
that  if  the  desired  change  is  evident  then  the  LMET  training,  hence  the 
program  can  be  surmized  as  being  effective.  There  is  at  least  one  con¬ 
cern  about  this  approach.  That  is,  the  LMET  graduate  needs  to  have  had 
sufficient  time  after  completion  of  the  training  to  have  developed  a 
leadership/management  behavior  pattern.  If  conducted  too  soon  after 
graduation  the  result  may  well  be  a  parroting  of  LMET  training  with  no 
real  demonstrable  behavioral  changes.  This  need  for  a  delay  or  time  gap 
between  data  gathering  necessitates  an  evaluation  that  could  be  quite 
lengthy  in  time. 

While  this  time  gap  may  suffice  for  a  full  scale  evaluation,  it 
would,  if  used  for  a  pilot  study,  create  problems.  By  extending  the 
length  of  time  to  conduct  a  pilot  study  its  usefulness  to  a  program 
manager  is  diminished.  In  a  dynamic  program  such  as  LMET,  an  evaluation 
needs  to  be  as  timely  as  possible  in  providing  information.  If  the  pilot 
study  is  overly  lengthy,  the  insights  provided  concerning  program  evalu¬ 
ation  may  no  longer  be  applicable  when  presented  to  the  program  manager. 

If  due  to  insufficient  time,  a  before  and  after  data  gathering 

approach  is  impractical,  then  another  method  must  be  employed,  that  of 

the  follow-up  or  post-test  data  collection.  Using  this  method  the  intent 

Is  to  gather  behavioral  data  after  completion  of  the  school  to  determine 

if  the  students  behavior  matches  that  which  the  program  is  designed  to 
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produce.  This  follow-up  or  post-test  method  while  the  least  rigorous  of 
those  mentioned  lends  itself  quite  well  to  a  pilot  study.  It  can  provide 
sufficient  data  to  identify  issues,  test  hypotheses  and  evaluate  a  meth¬ 
odology  for  conducting  a  full-scale  evaluation.  In  addition,  this 
approach  is  not  time  constrained,  is  extremely  flexible  and  easily  adapt¬ 
ed  to  the  resources  required  for  a  pilot  study.  A  drawback  to  this 
approach  is  that  data  gathered  from  a  student  in  a  program,  such  as  LMET, 
may  be  highly  opinionated  and  not  necessarily  indicative  of  actual 
behavior,  in  essence  only  a  self-report.  This  problem  can  be  overcome  by 
validating  the  subjects'  self-reported  data  with  additional  information 
gathered  from  persons  who  have  the  opportunity  to  closely  observe  the 
subjects'  behaviors.  Supervisors,  subordinates  and  peers  are  all 
potential  sources  of  this  validating  data. 

One  additional  consideration  deals  with  sample  size.  A  full-scale 
evaluation  in  order  to  be  considered  valid  needs  to  be  a  random  sample  of 
the  population.  Such  a  random  sample  can  then  be  assumed  to  statisti¬ 
cally  represent  the  whole  population.  When  considering  a  major  program, 
such  as  LMET,  this  random  sample  may  need  to  be  quite  large.  A  pilot 
study,  on  the  other  hand,  is  not  intended  to  be  a  substitute  for  a  full- 
scale  evaluation  and  need  not  use  a  random  sample  inorder  to  be  useful. 
A  sample  that  includes  a  cross  section  of  affected  people  from  the 
program  is  all  that  is  necessary.  The  intent  is  to  not  bias  the  pilot 
study  re-  suits  with  data  that  applies  to  only  a  small  segment  or  portion 
of  the  program  population.  The  sample  size  for  a  pilot  study,  when 
possible,  should  provide  a  broad  snapshot  covering  the  entire  spectrum  of 
the  program  under  evaluation. 
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C.  GUIDELINES  FOR  AN  LMET  PILOT  STUDY 

The  need  for  an  evaluation  of  the  LMET  Program  has  been  established. 
Furthermore,  the  arguments  in  favor  of  conducting  a  pilot  study  in  order 
to  provide  guidance,  as  to  how  to  conduct  that  evaluation  have  been  pre¬ 
sented.  The  remaining  action  is  to  formulate  the  guidelines  necessary  to 
conduct  the  LMET  Program  pilot  study. 

The  first  concern  was  who  should  be  included  in  the  sample? 
Obviously,  in  order  to  conduct  this  LMET  pilot  study,  information  needs 
to  be  gathered  on  LMET  school  graduates  as  to  behaviors  they  exhibit 
after  the  LMET  training.  This  then  identified  the  study  sample,  LMET 
gradu-  ates.  However,  any  data  collected  on  the  graduates  which  came 
from  the  graduates  themselves  would  need  to  be  validated  as  discussed 
earlier.  This  need  for  validating  data  also  directed  emphasis  towards  an 
LMET  graduates  immediate  supervisor  and  subordinate.  The  basic  data 
package  would  be  gathered  from  LMET  graduates,  with  validating  data 
gathered  from  their  immediate  supervisors  and  subordinates. 

Ideally,  a  graduate  who  attended  LMET  school  after  arriving  at  his 
present  command  would  potentially  provide  the  best  behavioral  change 
data.  Unfortunately  graduates  in  this  category  were  expected  to  be 
difficult  to  find  due  to  the  programs  scheduling.  This  scheduling 
established  the  normal  routine  of  attending  LMET  school  via  Permanent 
Change  of  Station  orders  [PCS],  that  is  before  arriving  at  their  command 
and  not  via  returnable-quota.  Temporary  Additional  Duty  orders  [TAD].  As 
a  result  of  this  procedure,  most  graduates  would  be  expected  to  have 
attended  LMET  prior  to  reporting  to  a  command,  hence  eliminating  the 
"before"  data. 
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Another  issue  in  this  question  of  who  to  interview  was,  which 
graduates?  The  LMET  Program  covers  eight  general  management  levels,  they 
are:  CO/XO,  Department  Head,  Division  Officer,  Master  Chief  Petty 

Officer  of  the  Command,  Leading  Chief  Petty  Officer,  Chief  Petty  Officer, 
Leading  Petty  Officer  and  Petty  Officer.  These  levels  could  be  further 
grouped  into  CO/XO,  Department  Head,  Division  Officer,  Chief  Petty 
Officer  and  First  Class  Petty  Officer  which  follows  the  normal  Navy  chain 
of  command.  Among  these  groups  the  lower  3  levels  of  Division  Officers, 
Chief  Petty  Officers  and  First  Class  Petty  Officers  are  the  most  numerous 
aboard  any  command,  hence  most  promising  as  to  accessabil ity,  thereby 
further  defining  the  pilot  study  sample. 

Limited  interviewer  resources  as  well  as  time  constraints  limited 
the  chance  of  obtaining  what  could  even  be  remotely  labeled  a  representa¬ 
tive  sample.  However,  it  needs  to  be  emphasized  that  this  was  not  to  be 
an  actual  evaluation,  but  rather  a  pilot  study.  Therefore  a  non-random, 
cross  section  of  these  LMET  graduates  would  be  sufficient.  In  this  way 
it  would  serve  to  evaluate  methodology,  identify  key  issues  as  well  as 
provide  some  crude  insights  into  LMET  effectiveness.  What  now  became 
important  would  be  sampling  a  sufficient  number  of  different  commands  as 
well  as  a  variety  of  billets  within  those  commands.  Since  operational 
commands  had  been  given  number  one  priority  [0LMSTEAD,T980]  for  LMET 
school  quotas  it  was  possible  to  further  restrict  the  sample  to  these 
commands.  The  most  likely  evidence  of  change  would  be  among  those 
commands  which  had  the  highest  degree  of  exposure  to  the  LMET  Program. 

Due  to  the  physical  location  of  the  Naval  Postgraduate  School  on  the 

west  coast,  a  practical  consideration  was  qiven  to  concentrate  on  west 

coast  commands.  Time  was  a  major  issue  with  travel  costs  providing  an 
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additional  incentive  to  remain  on  the  west  coast.  With  the  standardi¬ 
zation  of  the  LMET  curriculum  under  CNET,  it  was  felt  that  any  general 
conclusions  reached  using  west  coast  commands  would  most  likely  apply  to 
east  coast  commands  as  well.  With  the  geographic  location  narrowed  down, 
type  of  commands  on  which  to  concentrate  [operational  i.e.,  air,  surface, 
subsurface,  amphibious,  service  force],  and  target  billets  identified 
[Division  Officer,  CPO,  P01]  the  remaining  question  was  how  large  of  a 
sample  would  be  necessary?  The  issue  now  became  one  of  sampling  a 
sufficient  number  of  different  operational  commands  [air,  surface, 
submarine]  as  well  as  a  variety  of  billets  within  those  commands,  within 
the  limited  time  available.  The  aim  was  to  minimize  any  biases  peculiar 
to  either  type  of  command  [surface,  submarine,  aviation];  billet 
position  such  as  Division  Officer,  Leading  Petty  Officer  or  Chief  Petty 
Officer;  geographic  location  such  as  San  Diego;  or  specialty  area 
[weapons,  engineering,  supply,  etc.].  This  cross-sectional  sample  would 
be  sufficient  to  draw  conclusions  and  to  provide  certain  guidance  for  a 
future  full-scale  evaluation.  It  is  imperative  that  the  limitations  of 
this  "non-random  cross-sectional  sample"  be  appreciated  so  as  to  prevent 
this  research  from  being  mistaken  as  a  program  evaluation. 

That  leaves  the  remaining  question  of  how  to  best  gather  the  data. 
The  choice  of  methods  focused  on  two  possible  methods,  interviewing  or 
survey  questionnaire.  The  collection  of  data,  regardless  of  method,  in¬ 
volves  three  elementary  forms  of  human  activity:  observation,  partici¬ 
pation  and  empathy.  As  best  stated  by  R.  L.  Gorden: 


From  these  three  activities  all  of  the  methods  for  study¬ 
ing  hum  an  behavior  are  derived.  Within  this  framework,  inter¬ 
viewing  is  seen  as  one  specific  form  of  empathizing,  partic¬ 
ipating  and  observing  which  takes  place  between  two  people... 
in  this  context  the  questionnaire  [survey]  is  seen. .. .simply 
as  a  technique  for  extending  the  interview  in  which  the  re¬ 
searcher  participates  by  constructing  the  questionnaire. 
[G0RDEN,1980,p.5] 

Empathy  is  referring  to  feelings  with  another  person  or  understand¬ 
ing  how  that  person  feels  about  something.  Participation  refers  to  doing 
something  with  another  person  in  their  regular  ongoing  activities.  When 
taking  on  a  participatory  role  one  is  in  a  vantage  point  from  which  it  is 
possible  to  either  observe  another's  activity  or  to  introspectively  note 
one's  own  thoughts  and  feelings  as  related  to  that  situation.  Observa¬ 
tion  includes  any  sensory  perception,  not  only  visual,  of  cues  which  help 
a  person  understand  human  behavior.  With  these  three  concepts  in  mind 
let  us  examine  these  two  data  gathering  methodologies. 

The  first  of  the  two  methods  is  the  survey  questionnaire.  Observer 
participation  in  a  survey  with  the  respondent  can  only  be  dealt  with 
during  the  survey  construction  phase.  Actual  interaction  between 
observer  and  respondent  is  extremely  limited.  During  the  construction 
phase  it  is  possible  to  interject  ones  own  thoughts  as  to  what  is 
important,  what  needs  to  be  asked  or  which  issues  are  relevant.  Once 
constructed  and  administered,  no  further  participation  is  available  to 
the  "observer".  This  inflexibility  is  a  major  weakness  of  a  survey 
especially  in  a  pilot  study  where  the  meaningful  issues  may  not  be  known. 
Observation  for  a  survey  consists  of  looking  at  and  examining  responses 
written  on  the  survey  form.  While  perhaps  easily  quantifiable  data  may 
be  gathered,  data  that  can  be  easily  analyzed,  using  computer  programs, 
the  observations  are  limited.  The  circumstance  under  which  the  answers 
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were  written,  such  as,  the  respondents  emotional  level,  reaction  to  the 
survey,  workload  and  many  other  factors  are  lost  to  a  survey  administra¬ 
tor.  There  is  also  the  concern  of  whether  each  item  asked  on  the  survey 
was  fully  understood  by  the  respondent.  Educational  and  cultural  back¬ 
ground,  socio-economic  considerations,  as  well  as  the  environment  all 
have  influence  on  the  person  participating  in  a  survey.  There  is  the 
argument  that  this  prevents  the  observer  from  biasing  the  data.  Yet  in 
actuality  the  interviewers  biases  are  permanently  ingrained  in  the  survey 
itself  with  no  way  of  adjusting  to  different  respondents  short  of  re¬ 
writing  the  survey.  By  not  viewing  the  respondent  the  interviewer  looses 
that  personal  aspect  of  data  collection.  Any  empathy  must  be  transmitted 
via  the  survey  instrument  which  is  certainly  difficult  at  best. 

All  this  is  not  too  say  that  a  survey  does  not  have  distinct  advan¬ 
tages.  A  survey  can  be  administered  to  a  very  large  sample  in  a  re¬ 
latively  short  period  of  time  at  a  low  cost.  It  also  lends  itself  well 
to  computer  analysis.  For  data  that  is  easily  quantifiable  into  yes/no 
responses  or  to  scales  of  like/dislike,  a  survey  is  especially  appro¬ 
priate.  However,  when  gathering  data  on  a  highly  opinionated  subject 
where  there  are  no  obvious  choices  or  a  potentially  wide  variety  of  re¬ 
sponses  a  survey  can  be  useless. 

Another  consideration  is  that  survey  construction  is  an  exacting 
evolution.  A  good  survey  may  take  considerable  time  in  constructing, 
testing  and  revising.  The  problem  is  once  administered  it  is  difficult 
to  change  so  It  must  be  correct  the  first  time.  It  is  this  very  in¬ 
flexibility  that  limits  the  usefulness  of  a  survey  in  conducting  a  pilot 
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study,  especially  for  a  program  where  the  real  issues  may  not  yet  be 
known. 

The  second  method  considered  was  the  interviewing  process.  One 
obvious  advantage  to  this  method  is  the  opportunity  to  gather  data  from 
multiple  sources.  Not  only  are  there  the  verbal  responses  of  the  re¬ 
spondent  but  also  non-verbal  signals  such  as  facial  expressions,  hand  and 
body  motions  and  tone  of  voice  to  name  only  three.  All  of  these  obser¬ 
vations  can  be  used  by  the  interviewer  to  gain  a  deeper  insight  into  the 
responses  to  the  interview  questions.  This  multiple  observation  capa- 
bility  can  be  enhanced  by  team  interviewing,  where  one  person  questions 
and  concentrates  on  verbal  responses,  maximizing  the  flow  of  data  while 
the  other  person  concentrates  on  note  taking  and  non-verbal  responses. 

Participation  in  the  interview  is  another  advantage  over  surveys. 
The  observer  is  able  to  explore  fully  the  responses  given  to  questions  by 
solicitating  further  clarification  or  in  asking  additional  questions. 
This  flexibility  allows  the  interviewer  to  fully  explore  issues  as  they 
surface  while  ensuring  that  it  is  completely  understood.  This  specific 
aspect  fits  the  concept  of  a  pilot  study,  especially  the  added  flexi¬ 
bility  of  exploring  new  issues. 

Empathy  can  add  to  the  quality  of  the  data  gathered  via  interview¬ 
ing.  The  interviewer  in  being  able  to  develop  empathy  for  the  respondent 
during  the  course  of  an  interview  can  anticipate  probable  reactions  to 
questions  and  sense  how  the  respondents  feel  about  the  events  they  are 
relating.  This  can  add  a  qualitative  aspect  to  what  is  actually  being 
stated  thereby  further  enriching  the  data. 
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All  of  this  is  not  to  say  that  interviewing  does  not  have  drawbacks. 
It  is  a  time  consuming  process  and  costly  in  terms  of  money  and  manhours 
expended.  There  is  a  very  real  danger  of  the  interviewers  distorting  the 
data  with  their  own  biases,  perhaps  unknowingly.  Additionally,  it  takes 
a  certain  skill  to  establish  a  good  rapport  with  the  respondent.  Without 
it,  a  free  flow  of  information  is  not  possible. 

The  very  flexibility  of  the  interview  poses  another  problem  in  that 
it  *is  very  easy  to  get  sidetracked  into  irrelevant  issues  or  redundant 
detail  to  a  response.  The  interviewing  procedure  is  dependent  on  the 
free  flow  of  information  between  respondent  and  the  interviewer. 

The  interview  process  itself  can  inhibit  this  very  flow  of  inform¬ 
ation.  Interviewing  with  two  interviewers  on  one  respondent  may  infact 
be  intimidating  to  the  respondent.  If  a  tape  recorder  is  used  to  record 
the  verbal  data  it  may  similarly  intimidate  the  respondent. 

An  additional  obstacle  may  be  that  of  the  actual  person,  officer  or 
civilian,  used  in  conducting  the  interview,  especially  for  enlisted  re¬ 
spondents  discussing  their  supervisors.  The  Navy  officer  may  have  an 
advantage  of  being  an  insider  to  the  organization,  who  understands  the 
language  and  jargon  and  can  probably  empathize  easily  with  the  respon¬ 
dent.  If  in  uniform  he  will  present  a  familiar,  possibly  non¬ 
threatening  appearance  due  to  the  respondents  everyday  interface  with 
officers.  A  civilian  on  the  other  hand  may  hinder  the  flow  of 
communication  by  having  to  have  everyday  situations  and  acronyms  explain¬ 
ed.  There  is  also  the  possibility  of  misunderstanding  a  word  or  phrase 
that  is  Navy  jargon. 


54 


On  the  other  hand  the  civilian  interviewers  may  be  less  threatening 
in  that  they  do  not  represent  members  of  hiqher  Navy  management.  For  the 
Navy  respondent,  there  may  be  less  reluctance  to  voice  a  negative  opinion 
about  a  Navy  program  to  an  outsider  of  the  organization. 

0.  SUMMARY 

These  preceding  considerations  and  issues  concerning  a  pilot  study 
for  a  major  program,  such  as  LMET,  form  the  preliminary  step  in  its 
overall  evaluation  process.  While  not  all  encompassing,  each  of  these 
specific  issues  were  addressed  in  formulating  an  LMET  pilot  study  design. 
Any  major  program  requiring  evaluation  could  use  this  or  a  similar 
approach  in  order  to  develop  an  appropriate  direction  for  a  full  scale 
evaluation.  The  pilot  study  design  can  provide  decision  makers  with  the 
relevant  issues  and  guidance  to  better  optimize  their  resources. 


IV.  LMET  PILOT  STUDY  DESIGN:  A  DESCRIPTION 


A.  PREPARATION 

As  in  any  research  study,  a  pilot  study  being  no  exception,  a  plan 
of  attack  must  be  developed  before  one  packs  his  bags  and  heads  out  to 
gather  "data".  The  question  of  what  one  wants  or  seeks  to  find,  should 
be  clearly  established  from  the  start.  Without  this  being  established 
it  is  difficult,  if  not  impossible,  to  best  determine  the  course  of 
action  to  take.  Once  the  problem  identif ication  and  statement  is 
complete,  a  methodology  and  overall  design  strategy  for  the  study  can 
be  formulated. 

An  overall  evaluation  of  LMET  effectiveness  would  serve  nicely  as 
a  problem  statement  for  the  proposed  research.  After  all,  the  LMET 
Program  had  not  been  evaluated  and  effectiveness  as  related  to  im¬ 
proved  student  performance  could  provide  a  suitable  criteria  for 
measurement  purposes.  However,  this  was  not  to  be  the  case  for  at 
least  two  major  reasons,  that  of  a  time  constraint  and  previously  noted 
difficulties  in  measuring  effectiveness. 

First,  the  two  graduate  students  selected  would  have  only  6  months 
to  complete  the  study,  from  December  through  June.  While  sufficient 
time  to  commence  an  evaluation  presented  no  major  problems  through  June, 
both  were  scheduled  to  graduate  at  that  time  and  would  subsequently  re¬ 
ceive  seperate  Navy  orders  based  upon  service  needs.  Since  no  other 
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graduate  student  assets  were  available  with  sufficient  background 
experience  and  time  to  conduct  the  evaluation  it  was  agreed  to  conduct 
a  pilot  level  study  rather  than  a  major  overall  evaluation  of  LMET 
effectiveness. 

The  second  area  of  concern  centered  around  the  notion  of  effec¬ 
tiveness.  Previous  effectiveness  studies,  especially  in  the  area  of 
leadership  training,  in  both  the  civilian  and  military  sectors  are  few 
in  nunber,  yielding  inconclusive  results  as  a  whole.  It  has  been 
difficult  at  best  to  measure  situational  performance  of  personnel,  let 
alone  attempt  to  measure  an  improvement  or  increase  in  this  performance 
and  relate  that  to  a  specific  training  program  to  determine  its  overall 
effectiveness.  The  difficulty  in  determining  effectiveness  would  only 
be  compounded  by  the  aforementioned  insufficient  time.  The  probability 
of  obtaining  an  outcome  would  be  low  and  any  results  obtained  would 
lack  credibility. 

For  these  very  reasons  among  others  it  was  decided  to  conduct  a 
smaller  scale  pilot  study  using  interview  techniques  to  gather  data. 
The  hope  of  the  study  ••jould  be  to  obtain  useful  data  on  LMET  effect¬ 
iveness  to  better  design  a  much  larger  overall  LMET  evaluation. 

B.  A  PLAN  OEVELOPES 

In  December  1980,  a  recommended  study  design  was  received  from 
McBer  and  Company  based  upon  the  pilot  level  study  concept. 5  McBer 

5 In  the  letter  McBer  and  Company  reiterated  support  for 
efforts  to  conduct  an  LMET  evaluation.  [McBER  AND  COMPANY, 1981] 


recommended  the  strategy  of  a  combination  of  open.ended  interview 
questions  with  a  survey  instrument  to  be  completed  by  the  respondent 
immediately  after  each  interview.  Interviews  were  to  be  conducted  with 
LMET  graduates,  two  of  their  subordinates  and  an  immediate  super, 
visor.  Suggested  sample  questions  to  be  asked  for  each  level 
[graduate,  subordinate  and  supervisor]  were  included  from  McBer.  The 
questions  were  designed  to  concentrate  on  behaviors  [has  their  behavior 
changed?]  and  results  [does  it  make  any  difference  to  bottom  line 
measures?],  rather  than  reactions  [how  do  people  like  the  course?]  and 
knowledge  .  content  [what  did  they  learn?]. 6  in  this  way,  the  in 
depth  interviews  probing  for  specific,  behavioral  exanples  from  the 
LMET  graduates  could  be  validated  by  their  subordinates  and  supervisor 
in  an  effort  to  obtain  more  meaningful  data.  McBer  also  mentioned  a 
suggested  coding  scheme  for  the  interview  data  which  had  been  used  to 
evaluate  their  own  business  leadership  training  program.  McBer 
recommended  that  the  graduate  students  be  trained  in  interview  skills 
as  well  as  an  LMET  coding  scheme  for  scoring  interview  responses.  Their 
final  recommendation  was  to  send  both  researchers  through  the  LMET 
course  itself,  before  conducting  the  study. 

Through  December  and  January  the  pilot  study  preparations 
continued.  During  this  pre. interview  phase  the  major  task  would  be  to 

^McBer  states,  "...'Reactions'  are  best  addressed  immedi. 
ately  after  the  course  [which  Is  indeed  happening]  and  'knowledge  - 
content’  is  currently  being  evaluated  immediately  after  the  course  in 
response  to  requests  from  CNET." 


improve  interview  techniques  of  the  two  graduate  students  as  well  as  to 
develop  a  knowledgeable  background  on  LMET  and  general  Navy  leadership 
training. 

In  the  area  of  interview  techniques,  the  graduate  students  relied 
on  a  variety  of  sources  to  further  develop  their  skills.  Both  graduate 
students  were  surface  warfare,  line  lieutentant  commander  with 
similar  experience  in  management  practices  developed  in  at-sea  opera¬ 
tional  environments  for  a  combined  total  of  nearly  20  years  in  the 
Navy.  Additionally,  both  were  pursuing  Master  of  Science  degrees  in 
Management,  specifically  Human  Resource  Management  [HRM]  with  an 
emphasis  in  Organizational  Development  [O.D.]?.  Their  experience  as 
Naval  officers  as  well  as  their  knowledge  of  applied  and  theoretical 
concepts  of  management  would  serve  as  a  substantial  foundation  in 
developing  interviewing  skills. 

Prior  to  interviewing  both  graduate  students  attempted  a  2  day 
workshop  conducted  by  Captain  Phillip  Butler,  USN,  on  interviewing. 8 
The  workshop  concentrated  on  developing  and  planning  an  overall 
interview  strategy.  Stressing  the  biases  of  this  form  of  social 


?The  Human  Resource  Management  curriculun  at  the  Naval  Post¬ 
graduate  School  consists  of  core  courses  in  Economics,  Operations 
Research,  Systems  Analysis,  Management,  Accounting,  Computers  and 
Statistics  followed  by  sub  speciality  courses  in  Organizational  Theory, 
Educational  Design,  Labor  Relations  and  Personnel  Management, 
Organizational  Development,  Individual  and  Group  Processes  as  well  as 
Public  Administration. 

^Captain  Butler  holds  a  Doctorate  in  Sociology  from  the 
University  of  California,  San  Diego.  He  is  well  known  for  his  active 
use  of  interviewing  as  a  viable  means  of  gathering  data  in  both 
military  and  civilian  organizations.  He  was  a  co. advisor  for  the  pilot 
study. 
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inquiry,  he  introduced  specific  interview  methods  and  skills  available 
as  well  as  thoughts  on  their  effective  use.  At  the  conclusion  of  the 
workshop  participants  were  given  the  opportunity  to  team  interview  on  a 
variety  of  subjects  with  subsequent  feedback  on  their  effectiveness . 
Using  the  concepts  developed  from  this  workshop,  actual  interviews  were 
conducted  for  practical  experience.9  The  next  area  of  concern  was 
now  turned  towards  the  LMET  concept  and  the  actual  course.  Research  on 
the  development  of  Navy  leadership  training.  Dr.  McClelland's  concepts 
of  achievement  and  competency  notions,  McBer’s  LMET  study,  and  course 
content  of  LMET  [all  previously  mentioned],  were  undertaken  to  formu¬ 
late  a  satisfactory  base  of  knowledge.  While  there  was  no  time  to 
actually  attend  the  LMET  course  due  to  other  graduate  course  require¬ 
ments,  a  trip  was  made  to  the  LMET  School  in  Coronado,  California. 
During  that  visit  both  graduate  students  were  allowed  to  sit  in  on  the 
various  levels  of  classes  as  well  as  talk  to  the  curriculum  director 
and  a  nunber  of  LMET  instructors.  Literature  on  course  content,  sample 
course  schedules  as  well  as  general  LMET  information  was  obtained  and 
later  reviewed.  In  mid  January,  the  research  background  on  LMET  as 
well  as  the  interview  training  was  completed,  a  meaningful  plan  of 
attack  could  now  be  developed  complete  with  a  schedule  of  milestones. 

9LCDR  D.  Vandover,  one  of  the  two  pilot  study  researchers, 
using  interview  concepts  from  CAPT  Butler's  workshop  as  well  as 
additional  research,  designed  and  conducted  a  1  day  interview  workshop 
which  was  given  to  a  class  of  graduate  students  at  the  Naval 
Postgraduate  School.  Later  both  students  received  practical  training 
In  conducting  Interviews  while  Involved  In  consulting  with  a  local 
business  in  Monterey,  California.  These  Interviews  were  conducted  with 
eighteen  owners/ employees  with  wide  educational  backgrounds,  and 
included  grouping,  categorizing  and  analyzing  a  large  amount  of  data. 
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C.  THE  PLAN 

A  plan  to  conduct  the  pilot  study  was  proposed  to  and  approved  by 
Dr.  Reuben  Harris.  Interviews  were  to  commence  later  in  January  and 
continue  through  the  end  of  March,  with  the  hope  of  obtaining  at  least 
20  sets  of  interviews.  April  was  to  be  set  aside  to  analyze  the  data 
while  May  was  to  be  used  to  write  the  final  report.  The  major  sections 
of  McBers  recommended  pilot  study  design,  including  the  questions, 
would  be  utilized  with  three  exceptions:  first,  the  survey  after  the 
interview  would  not  be  used.  Next,  McBer's  coding  scheme  for  the 
interview  data  would  not  specifically  be  employed,  thereby  eliminating 
this  additional  training  requirement;  and  finally,  the  graduate 
students  would  not  attend  the  ten  day  LMET  course.  ^ 

Dr.  Harris  obtained  a  point  of  contact  to  assist  the  graduate 
students  in  scheduling  the  commands  to  be  interviewed  using  the  Human 
Resources  Management  Center  [HRMC]  in  San  Diego,  California,  and  the 
Human  Resources  Management  Detachment  [HRMD]  in  Alameda,  California, 
for  the  commands  in  those  areas  respectively.  It  was  left  up  to  the 
graduate  students  to  contact  the  center  and  detachment  to  work  out  the 
final  details.  Additionally,  the  cognizant  officer  on  the  staff  of  the 


‘®A  list  of  the  actual  questions  used  during  the  study  is 
contained  in  Appendix  A. 

^All  three  design  recommendations  appeared  to  be  good 
suggestions,  however,  the  report  was  not  received  until  mid  January. 
This  left  insufficient  time  to  fully  develop  a  survey,  arrange  for 
additional  training  in  McBer's  coding  scheme  and  attend  two  weeks  of 
LMET  school.  In  addition,  due  to  the  small  sample  size,  it  was  felt 
that  the  survey  would  provide  little  additional  insight  to  this 
particular  study.  All  three  recommendations  deserve  thoughtful 
consideration  as  part  of  a  large  scale  evaluation. 
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Commander,  Pacific  Fleet,  [CINCPACFLT],  was  notified  of  details 
concerning  the  study.  The  types  of  commands  and  levels  of  LMET  gradu¬ 
ates  to  be  interviewed  were  not  specified  and  were  left  to  the  judge- 
ent  of  the  graduate  students. 

It  was  decided  by  the  graduate  students  to  try  to  interview  a 
wide-ranging  sample  of  operational  [as  opposed  to  shore  and  support] 
commands  in  terms  of  warfare  specialties;  that  is  air,  submarine  and 
surface,  with  emphasis  on  the  latter  due  to  their  familiarity  and 
experience.  Since  the  emphasis  would  be  on  surface  ships  the  sample 
would  be  further  divided  into  carriers,  cruisers  and  destroyers, 
amphibious,  auxiliary  support  and  miscellaneous.  Each  of  the  three 
major  Pacific  Fleet  geographical  areas  would  be  sampled,  San  Diego, 
California;  Alameda,  California;  and  Pearl  Harbor,  Hawaii.  Among  the 
selected  commands,  it  was  decided  to  interview  LMET  graduates  in  three 
specific  billet  levels,  each  corresponding  to  LMET  levels  of  petty 
officer  [E-6],  chief  petty  officer  [E-7  thru  E-9],  and  division 
officer  [0-1  thru  0-4].  Department  head  graduates  as  well  as  executive 
and  commanding  officers  would  not  be  interviewed,  except  as  immediate 
supervisors  to  the  LMET  graduates.  On  January  29,  1981,  the  first  in¬ 
terviews  for  the  pilot  study  were  conducted  onboard  a  surface  ship  in 
Alameda,  California.  Nearly  two  months  later  the  last  interview  would 
be  completed  onboard  another  surface  ship  in  Pearl  Harbor,  Hawaii. 

Actual  selection  of  the  commands  to  be  Interviewed  was  left  to  the 
discretion  of  the  HRMC  and  HRMD.  The  only  guidance  given  was  that  the 
wider  the  variety  of  operational  commands  the  better.  Undoubted  a  major 
consideration  was  which  commands  were  Inport.  Any  other  criteria  that 
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the  HRMC/D  considered  in  selecting  commands  is  not  known,  however, 
there  remains  a  possibility  that  other  selection  criteria  may  have  been 
used  and  may  affect  the  data. 

One  to  two  weeks  prior  to  the  interviews  the  point  of  contact  in 
the  desired  area  was  called  and  asked  to  nominate  or  identify  a  command 
for  each  of  the  two  days  the  interviewers  were  planning  to  be  in  the 
area.  Once  the  commands  had  been  identified,  [points  of  contact  at  the 
command  and  telephone  numbers  were  supplied  by  the  HRMC  and  HRMD],  the 
interviewers  would  contact  the  command,  usually  through  the  executive 
officer,  to  explain  the  proposed  visit  and  confirm  a  date  and  the 
commands  actual  location.  This  selection  procedure  continued  smoothly 
throughout  the  interview  phase  with  relatively  few  problems.  It  was 
easy  to  tell  if  the  HRMC  or  HRMD  had  actually  contacted  the  units  or 
just  identified  them  based  upon  an  employment  schedule,  commands 
personally  contacted  by  the  HRMC  and  HRMD  were  far  more  enthusiastic 
about  the  proposed  visit  than  were  those  not  contacted.  While  a 
considerable  amount  of  extra  time  was  necessary  to  "convince  and  sell 
the  visit"  to  unaware  commands,  no  one  ever  refused  to  visit. 

The  initial  telephone  contact  with  the  commands  by  the 
interviewers  generally  lasted  15-30  minutes.  This  was  a  sufficient 
length  of  time  to  explain  the  specific  details  of  the  visit.  The  visit 
was  explained  as  a  study  on  LMET  effectiveness  for  the  specific  purpose 
of  completing  a  Masters  Thesis  at  the  Naval  Postgraduate  School.  Upon 
realizing  that  the  interviewers  were  Navy  surface  warfare  officers, 
instant  credibility  seemed  to  be  established,  resulting  in  an  Informal, 
open  door  policy  for  the  visit.  One  executive  officer  remarked,  "Hey, 
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anything  you  guys  need - it's  good  to  see  the  Navy  using  its  own  like 

this."  The  executive  officers  were  informed  of  the  desire  to  interview 
3-5  LMET  graduates  [within  the  three  levels  previously  discussed], 
along  with  their  immediate  supervisor  and  two  subordinates.  They 

were  told  that  the  interviews  for  the  LMET  graduates  would  last 
approximately  one  hour,  while  subordinate  and  immediate  supervisor 
interviews  would  only  take  a  total  of  30  minutes.  Initially,  the 

executive  officers  were  allowed  to  set  up  their  own  schedule  for  the 
days  interviewing.  However,  as  the  study  progressed  a  schedule 
developed  by  one  of  the  commands  became  the  recommended  example  that 
executive  officers  were  encouraged  to  follow.^  The  only 

requirements  stipulated  by  the  interviewers  was  the  need  for  two 
separate  spaces  or  rooms  [relatively  quiet  if  possible],  which  would  be 
used  to  actually  conduct  the  interviews.  The  executive  officer  was 
then  informed  of  the  interviewers  desires  to  wear  civilian  clothes  as 
part  of  the  study,  when  applicable,  no  one  refused: 

Actual  selection  of  the  LMET  graduates  at  each  command  was  left  to 
the  discretion  of  the  executive  officer.  It  is  not  known  how  the 
selection  process  actually  occurred  in  each  instance,  however,  it  is 
known  that  some  graduates  volunteered  while  others  "were  volunteered" 

^After  interviewing  at  several  commands  the  two  subordinate 
requirement  was  changed  to  include  only  one  immediate  subordinate. 
This  resulted  in  a  time  savings. 

^McBer  had  suggested  1.5  hours  for  LMET  graduates  and  45  minutes 
for  subordinates  and  supervisors.  As  the  study  progressed,  actual 
interview  times  for  LMET  graduates  rarely  lasted  60  minutes,  with  sub 
ordinate  and  supervisor  interviews  easily  completed  within  15  minutes. 

^The  sample  schedule  used  is  contained  in  Table  2. 
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TABLE  2 

SAMPLE  INTERVIEW  SCHEDULE 


The  following  personnel  are  scheduled  for  interviews  on  (date)  at 
the  indicated  times.  Interviews  will  be  held  in  the  following  two 
locations: 


to  participate.  The  executive  officer  was  not  requested  to  specifical¬ 
ly  nominate  certain  levels  of  individuals  since  the  totals  among  the 
three  levels  remained  fairly  equal  throughout  the  entire  study. 

Several  days  prior  to  the  actual  visit  the  commands  were  again 
contacted  to  remind  them  of  the  time  and  date  of  the  visit  as  well  as 
to  recheck  the  actual  location  of  the  ship.  While  problems  were  few 
and  far  between,  they  were  all  resolved  during  this  final  contact.  On 
one  occasion,  a  forced  cancellation  by  a  command  was  required  due  to  an 
upcoming  inspection.  Using  personal  contacts  by  the  interviewers, 
another  command  was  substituted  causing  no  problems. 

All  interview  visits  except  one  were  scheduled  during  the  normal 
work  week,  with  the  vast  majority  occurring  on  Thursdays  and  Fridays. 
While  the  weekends  were  certainly  more  convenient  to  the  interviewers 
due  to  graduate  course  studies,  they  were  incredibly  inconvenient  for 
the  commands  for  obvious  reasons.  The  one  exception  occurred  through 
special  arrangement  over  a  weekend  after  it  was  learned  that  a 
satisfactory  sample  [five  graduates,  supervisors,  and  subordinates] , 
could  be  obtained  within  the  duty  sections  over  a  two  day  period. 

Arrival  at  the  commands  on  the  day  of  the  interviewing  was  usually 
between  0745-0815.  Although  the  individual  commands  generally  start 
their  work  day  earlier,  it  was  felt  that  it  would  be  more  convenient 
and  less  hectic  to  arrive  after  officer's  call,  giving  the  executive 
officer  as  well  as  the  ship  a  chance  to  start  their  day  before  our 
intrusion  -  the  commands  agreed  unanimously.  On  every  occasion  the 
quarterdeck  watch  had  expected  the  arrival  and  the  Interviewers  were 
taken  to  the  executive  officer.  After  introductions  and  the 


interviewing  plan  had  been  discussed  the  executive  officer  would 
typically  ask,  "..So  what  have  you  found  out  about  LMET  that  we  should 
know?"  On  several  occasions  the  interviewers  were  asked  to  meet  the 
commanding  officer  where  he  also  expressed  an  interest  in  learning  what 
had  been  found  to  date.^ 

D.  THE  INTERVIEW  PROCESS 

Interviews  were  first  conducted  with  the  LMET  graduates  using 
both  interviewers.  One  interviewer  would  question  the  respondent  while 
the  other  interviewer  would  passively  take  notes  as  well  as  handle  the 
tape  recorder.  The  respondent  was  introduced  to  the  interviewers  and 
made  to  feel  relaxed.  He  was  told  specifically  about  the  study  and 
assured  of  confidentiality  concerning  his  responses.^  He  was  told 
that  if  he  was  to  be  quoted  there  would  be  absolutely  no  mention  of 
either  his  name  or  his  command.  His  permission  was  solicited  to  allow 
the  use  of  the  tape  recorder  during  the  interview.  Without  exception, 
permission  was  always  received,  allowing  for  a  complete  taping  of  all 
interviews.^  He  was  further  asked  not  to  mention  any  names  of 

individuals  during  the  interview. 

1 5 

Commanding  officers  and  executive  officers  of  the  commands 
interviewed  were  generally  interested  in  any  information  concerning 
LMET.  The  interviewers  were  careful  to  avoid  any  evaluative  remarks 
under  the  guise  that  it  would  not  be  known  until  all  interviews  had 
been  completed  with  the  data  analyzed. 

^All  51  interviewed  LMET  graduate'  were  male  except  in  one 
exception  a  female  division  officer.  The  use  of  the  masculine  pronoun 
throughout  the  research  denotes  the  entire  sample  including  this 
exception. 

^All  tapes  contain  an  interview  number  as  well  as  the  commands 
name.  Respondents  are  identified  only  as  to  their  LMET  level  [petty 
officer,  chief  petty  officer  or  officer]  as  well  as  their  relationship 
[graduate,  subordinate  or  supervisor]. 
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The  graduates  were  all  asked  the  17  questions  [Appendix  A]  origin- 

18 

ally  recommended  by  McBer.  Additionally,  they  were  asked  the 

following  four  questions: 

1.  How  long  ago  did  you  graduate  from  LMET  school? 

2.  Did  you  attend  LMET  school  prior  to  arriving  at  this 
command? 

3.  Would  you  send  anyone  to  LMET  school?  Why? 

4.  Is  there  anything  else  you  would  like  to  tell  us  con- 
concerning  LMET? 

The  interviews  with  the  graduates  proceeded  smoothly.  The  only 
minor  problem  was  a  confusion  between  LMT  and  LMET.  Seven  respondents 
identified  by  their  commands  as  LMET  graduates  had  actually  attended 
LMT  school.  These  interviews  were  not  included  in  the  study  results 
reported  and  were  terminated  as  soon  as  this  fact  was  uncovered. 

Interviews  with  the  graduates  immediate  subordinate  and  supervisor 
were  conducted  after  the  graduate's  interview,  usually  later  that 
afternoon.  In  this  way  they  could  be  used  to  validate  the  graduates 
remarks.  Interviews  were  conducted  by  a  single  interviewer,  with  two 
interviews  ongoing  simultaneously  in  separate  rooms.  Identical  intro¬ 
ductions  were  made  as  had  been  done  with  the  LMET  graduates. 
Confidentiality  was  promised  and  respondents  were  asked  to  omit  names 
using  only  "him"  or  "he"  to  identify  the  graduate.  Permission  was 
obtained  in  every  case  to  record  the  interviews  on  tape. 

]8In  some  instances  several  LMET  stimulus  questions  were 
specifically  omitted  if  the  interviewers  believed  they  had  already  been 
answered  in  discussing  previous  questions. 
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The  subordinates  as  well  as  the  supervisors  were  all  asked  the 
four  questions  recommended  by  McBer.  Additionally,  they  were  asked  the 
following  four  questions  [note  the  difference  in  questions  2  and  3 
depending  upon  the  respondent]: 

[Both]  1.  What  do  you  like  or  admire  most  about  his  [the 
graduates]  performance  onboard? 

[Subordinate]  2. a  Would  you  work  for  him  again?  Why? 

[Supervisor]  2.b  Would  you  work  for  him  [if  roles  were  reversed]? 

Why? 

[Subordinate]  3. a  Would  you  like  to  go  to  LMET  school?  Why?^9 

[Supervisor]  3.b  Would  you  send  anyone  to  LMET  school?  Why? 

[Both]  4.  Is  there  anything  else  you  would  like  to  say 
concerning  this  individual  or  LMET  school? 

In  some  of  the  earlier  interviews  with  these  individuals  it  was 
noted  that  they  did  not  proceed  as  smoothly  as  had  the  graduates  inter¬ 
views.  There  was  a  tendency  on  the  subordinates  part  to  remain  intro¬ 
verted  and  to  assume  that  the  interviewers  were  trying  to  personally 
grade  or  evaluate  the  graduate.  Their  perception  seemed  to  be  that 
their  information  could  have  detrimental  con sequences  on  either  them¬ 
selves  or  the  LMET  graduate.  Once  this  was  noticed,  early  in  the  study, 
an  effort  was  made  prior  to  the  interview  to  explain  to  them  that  the 
information  they  would  give  would  be  used  only  to  categorize  the  LMET 
graduate  among  other  LMET  graduates.  Interviews  with  supervisors 
proceeded  smoothly  with  no  problems. 

19If  it  was  found  that  the  subordinate  had  gone  to  LMET  school, 
then  question  3.b.  was  substituted. 
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At  the  conclusion  of  the  day,  after  all  interview  had  been 

completed,  the  interviewers  then  met  to  discuss  the  days  events. 
Preparations  for  the  next  interviews  were  begun  using  any  new  lessons 
learned  from  that  days  activities. 

The  interview  process  as  discussed  above  was  repeated  through 
March  in  a  fairly  methodical  fashion.  In  the  end,  the  pilot  study 
sample  size  consisted  of  51  LMET  graduate  interviews  evenly  divided 
among  the  three  junior  levels  of  LMET  training  [LPO,  CPO  and  DO].  The 
sample  represented  13  operational  commands  within  the  3  warfare  areas 
in  three  Pacific  Fleet  geographical  locations. 20 

During  the  next  four  weeks  the  tedious  process  of  analyzing  the 
data  was  undertaken.  Responses  to  each  question  were  carefully 

analyzed  using  the  notes  and  tapes  taken  during  the  interviews.  All 
data  was  team  analyzed  using  both  students  rather  than  each  separately 
analyzing  the  data  twice.  A  coding  scheme  was  developed  for  each 

question  prior  to  analyzing  the  data,  in  an  attempt  to  categorize  the 

data  into  a  somewhat  more  usable  format.  Although  extremely  time 
consuming,  the  interviews  analyzed  using  the  LMET  [McBER]  competencies 
produced  an  enormous  amount  of  information  concerning  both  the  LMET 
program  in  general  as  well  as  specifics  on  LMET  graduates. 


20summary  results  of  the  sample  are  contained  in  Table  3. 
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TABLE  3 

PILOT  STUDY  SAMPLE 


I.  BREAKDOWN  OF  SAMPLE  BY  COMMAND  LOCATION 


Type  of 

Graduate 

Interviewer' s 

Command 

Location 

Rate/Rank 

Appearance 

Service  Force  Ship  [AFS] 

A1  ameda 

SK1/ENS/BM1/RM1 

Civil  ian* 

Amphibious  Ship  [LPD] 

Alameda 

LTJG/ET1/ENS/SKC 

Civilian 

Nuclear  Submarine  [SSN] 

San  Diego 

RMC/ST1/IC1/MMC/ETCS 

Civilian 

Small  Craft  Unit 

San  Diego 

BMC/SK1/BM1/HM1/ENCS 

Civilian 

Service  Force  Ship  [AOR] 

A1  ameda 

MSC/RMC/ EMC/LTOG/HT  C 

Uniform 

Service  Force  Ship  [AFS] 

A1  ameda 

ENS/DP1/QMCS/RMC 

Uniform 

Submarine  Rescue  Ship  [ASR] 

San  Diego 

LT/ENCS/EN1 

Uniform 

Aviation  Squadron  [RVAW] 

San  Diego 

LT/LCDR/LT/ENS 

Uniform 

Aircraft  Carrier  [CV] 

A1  ameda 

MM1/ENS/AG1 

Uniform 

Tender  [AR] 

San  Diego 

BM1/HTC/ET1/BM1/HTC 

Uniform 

Destroyer  [OD] 

San  Oiego 

DSl/GMGT/OSC/GSEC/ENS  Civilian 

Fast  Frigate  [FF] 

Cruiser  [CG] 

Pearl  Harbor  ENS/ENS 

Uniform 

Pearl  Harbor  ENS/LT 

Uniform 

*23  interviews  in  civilian 

II.  BREAKDOWN  OF  SAMPLE  8Y 

clothes,  28 

PAYGRADE 

in  uniforms 

Paygrade  of  LMET  Graduate 

E6  E7 

E8  E9  01  02 

03  04 

(Rank) 

P01  CPO 

SCPO  MCPO  ENS  LTJG 

LT  LCDR 

TOTALS 

18  13 

4  0  9  2 

4  1 

LMET  Level  of  Graduate 

E6 

E7-E9 

01,04 

Number  in  Sample  (X) 

18  [35.3] 

17  [33.3] 

16  [31.4] 

TABLE  3  [con't] 


III.  BREAKDOWN  BY  WARFARE  SPECIALTY/RATE 


11  Surface  Warfare  Officers 
1  Supply  Corp  Officer 
3  Aviation  Officers 
1  Aviation  Maintenance  Officer 


Supply  Dept. 

Medical 

Weapons/ Deck 

Aviation 

Engineering 

Operations 

3  SK’s 

1  HM 

5  BM's 

1  AG 

1  IC 

4  RM's 

1  MS 

1  ST 

2  MM's 

3  ET's 

1  DP 

1  GM 

3  EN's 

1  qM 

1  EM 

1  DS 

3  HT's 

1  OS 

— 

- 

— 

- 

_J_  GSE 

— i — 

5 

1 

7 

1 

11 

10 

IV.  BREAKDOWN  BY  TIME  SINCE  GRADUATION  FROM  LMET  SCHOOL 


Officer  CPO  POl 


Less  then  6  mo.  4 

6-12  mos.  5 

1_2  years  7 

Over  2  years  0 


6  9  19  [37.3X] 

7  3  15  [29.4X] 

4  4  15  [29.4XJ 

0  2  2  [  3.9X] 

ST  [100. OX] 
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V.  ANALYSIS  AND  FINDINGS 


The  data  gathered  from  the  pilot  study  can  be  grouped  into  two 
broad  areas.  First  is  the  data  indirectly  gathered  through  observing 
and  participating  in  the  interview  process,  that  is,  information  on  and 
resulting  from  the  methodology  used.  Second  is  the  actual  content  of 
the  interviews,  that  is,  data  concerning  the  LMET  graduate  to  the  LMET 
Program.  These  data  findings  can  be  used  to  judge  the  value  of  the 
pilot  study  technique  as  a  preliminary  evaluation  plan  for  major 
program,  specifically  LMET  in  this  case,  prior  to  committing  extensive 
resources  towards  a  full-scale  evaluation. 

A.  INTERVIEW  ANALYSIS 

Before  discussing  actual  interview  content  and  LMET  findings,  let 
us  look  first  at  the  process  used  to  specifically  gather  this 
information.  There  exists  many  advantages  as  well  as  disadvantages 
[discussed  earlier],  in  using  interviews  to  gather  data.  The  quality 
of  data  [both  in  depth  and  range]  obtained  from  interviews  is  normally 
offset  by  the  amount  of  time  required  in  using  this  method.  Once  the 
interview  method  has  been  examined  in  relation  to  this  study,  the  LMET 
content  findings  will  be  presented. 

1 .  Interviewing:  A  Time  Consuming  Process 

The  interviewing  process  was  extremely  time  consuming,  sub. 
stantiating  previous  research  claims  to  this  fact.  Each  LMET  graduate 
was  team  Interviewed  for  approximately  45  minutes  by  an  active  and 
passive  observer  for  a  total  expenditure  of  2.25  man. hours.  Each, 


immediate  subordinate  and  supervisor  to  the  LMET  graduate  was  inter, 
viewed  for  approximately  15  minutes,  one. on. one  for  a  total  expenditure 
of  1.0  man.hour.  A  simple  arithmetic  computation  reveals  that  3.25 
man-hours  were  expended  per  LMET  graduate  set  of  interviews. 
Theoretically,  this  times  the  51  interview  sets  equals  a  total  of 
165.75  man-hours.  In  actuality,  due  to  interviews  lasting  longer  than 
45  minutes,  approximately  180  man-hours  were  expended  in  data 
collection.  This  only  accounts  for  actual  time  expended  in 

interviewing  and  does  not  include  any  travel,  or  set-up  time  for  the 
interviewers  nor  time  spent  arranging  for  the  interviews.  It  is  worthy 
to  mention  that  considerable  time  and  travel  money  was  conserved 
through  interviewing  several  graduates  per  command  [3  to  6  graduates 
with  5  the  ideal  nimber  for  1  day].^ 

This  lengthy  expenditure  of  time  in  having  the  respondents 
full  attention  was  a  primary  advantage  in  obtaining  a  wide  range  and 
depth  of  information.  It  allowed  ample  opportunity  to  exploit  the 
respondents'  desire  to  voice  their  own  feelings  concerning  the  LMET 
training,  as  well  as  specifically,  how  LMET  was  seen  to  be  emphasized 
at  that  command.  Those  respondents  that  were  extremists  towards  the 
training  were  quick  to  point  out  its  weaknesses  or  strengths  while 
those  that  were  neutral  presented  a  more  toned  down  version.  The  end 
result  was  a  good  overall  picture  of  how  LMET  graduates  behaved  and 
felt  about  their  training  within  the  sample. 


sample  interview  schedule  is  contained  in  Table  2. 


74 


2.  In  Depth  Process  Analysis:  The  LMET  Interview 

This  section  contains  general  remarks  on  the  LMET  interview 
process  followed  by  examining  five  specific  areas  concerning:  interview 
flexibility,  the  interview  schedule,  the  interview  environment,  the 
issue  of  military  vs  civilian  interviewers,  and  finally  comments 
concerning  interview  introduction.  The  data  consists  of  69.25  hours  of 
taped  conversation  in  a  leading  question,  free  answer  format.  As  such, 
the  information  is  considered  to  be  soft-data,  in  that  it  is  difficult 
to  quantify.  Often  it  was  seen  that  two  people  can  use  very  similar 
terms  to  describe  the  same  subject  yet  mean  very  different  ideas.  Non- 
verbalized  expressions  such  as  tone  of  voice,  hand  and  body  movements, 
and  facial  expressions  served  to  emphasize  points  and  at  times 
contradict  what  was  actually  being  stated.  This  caused  the  interviewer 
to  explore  deeper  while  taking  written  notes  concerning  the 
respondent's  attitude  and  behavior.  This  resulted  in  data  consisting 
of  not  only  the  taped  conversations,  but  also  the  notes  and  feelings  of 
the  interviewers.  Even  after  51  interviews  it  was  still  possible  to 
vividly  remember  each  respondent  and  their  general  attitude  while 
listening  to  the  tapes  as  much  as  four  months  after  the  interview.  In 
addition  to  the  interview  questions,  the  respondent  was  given  the 
opportunity  to  make  any  additional  consents  that  might  be  pertinent  or 
of  felt  importance.  This  resulted  in  a  sizable  amount  of  not  easily 
catagoried  responses  that  added  to  the  total  information  obtained. 

The  subjectiveness  of  this  soft  data  was  further  accented  by 
the  Interviewer' s  own  biases  towards  the  LMET  Program.  This  bias  tend¬ 
ed  to  cause  an  interviewer  to  probe  deeper  into  certain  areas  where  the 
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respondent  was  reinforcing  the  biases  of  the  interviewer  and  perhaps 
cutting  off  or  playing  down  other  those  in  opposition.  As  the 
interviewing  progressed,  the  interviewers  started  to  observe  what  were 
thought  to  be  trends  and  may  have  tended  to  guide  respondents  into 
confirming  those  ideas  or  trends.  Oppossing  these  tendencies  were  the 
interviewer's  knowledge  of  their  own  biases  and  a  professional  interest 
in  suppressing  them  as  much  as  possible, 
a.  Interview  Flexibility 

One  of  the  early  advantages  that  became  apparent  in  using 
the  interview,  as  opposed  to  a  survey,  was  its  flexibility  in  gathering 
data.  The  interviewer  was  able  to  control  the  direction  of  the  infor. 
mation  flow  through  questions  and  requests  for  clarification  of 
statements.  It  quickly  became  obvious  that  the  interview  questions  had 
a  natural  progression  that  made  asking  some  unnecessary  as  the  answer 
would  be  given  in  discussing  a  previous  question.  This  resulted  in 
rearranging  the  questions  in  this  natural  order  and  merely  encourag¬ 
ing  the  respondent  to  continue  until  all  of  the  desired  data  had  been 
obtained.  In  not  breaking  the  respondent's  train  of  thought,  the 
interviewing  process  progressed  smoothly  and  quickly,  allowing  ample 
opportunity  for  the  respondents  to  voice  their  thoughts  on  any  given 
topic  or  question.  While  this  did  allow  for  the  respondent  to  answer 
each  question  fully,  it  also  led  to  some  detailed  examples  and 
explanations. 
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The  opportunity  of  interaction  between  interviewer  and 
respondent  ensured  that  the  interviewer  was  correctly  interpreting  re. 
sponses.  In  rephasing  further  questions,  and  asking  for  clarification 
of  specific  responses,  the  interviewer  had  ample  opportunity  to  clear 
up  any  misconceptions  or  hazy  areas  identified  by  the  respondent  during 
the  interview.  Similarly  a  distinct  advantage  of  the  interview  was  the 
ability  to  rephase  questions  to  ensure  the  respondent  understood  what 
was  being  asked.  This  exchange  of  ideas  led  to  the  rewriting  of 
several  questions  that  were  ambiguous  or  unclear  as  to  what  they  were 
asking.  It  was  discovered  that  what  seemed  to  be  a  perfectly  clear 
question  to  either  interviewer  was  often  not  understood  by  a  respondent 
of  a  different  educational  or  cultural  background. 

As  a  final  note  on  this  subject,  the  very  flexibility  of 
the  interview  process  required  the  undivided  attention  of  the 
interviewer.  This  results  in  a  tendency  of  the  interviewer  to  "burn 
out"  after  several  consecutive  days  of  interviewing.  The  interviews 
were  conducted  weekly  over  a  two  day  period,  normally  a  Thursday  and 
Friday.  If  a  full  five  days  of  interviewing  were  conducted  there  may 
have  been  a  difficulty  for  interviewers  to  maintain  their  interest  and 
enthusiasm  for  the  project.  The  consequence  is  obviously  a  risk  of 
gathering  lower  quality  data  at  the  end  of  a  week  as  compared  to  the 
beginning. 

b.  Interview  Schedule 

An  interviewing  schedule  was  developed  and  used  soon  after 
the  data  gathering  phase  had  begun.  For  the  first  two  commands  no 
schedule  was  provided,  only  a  description  of  what  was  necessary, 


namely  to  interview  four  to  six  LMET  graduates  as  well  as  an  immediate 
subordinate  and  supervisor  for  each.  The  first  command  devised  and 
promulgated  a  schedule  [as  well  as  a  Plan  of  the  Day  [POD]  note],  that 
worked  so  well  [see  Table  2]  that  it  was  recommended  for  use  to  all 
subsequent  commands.  The  use  of  this  standardized  schedule  allowed  the 
interviewers  to  develop  a  routine  that  minimized  wasted  time,  and 
increased  efficiency  in  conducting  the  interviews.  Additionally,  most 
command  XOs  appeared  grateful  to  receive  a  schedule  format  in  which 
they  could  merely  insert  names.  It  reduced  the  administrative  burden 
to  the  commands  as  well  as  completely  satisfying  the  interviewer's 
needs.  The  sample  schedule  also  minimized  disruptions  for  their  crew 
by  promulgating  beforehand,  an  interview  schedule  complete  with  inter¬ 
view  location  and  time. 

The  schedule  that  was  developed  by  the  first  command  had 
all  of  the  LMET  graduates  interviewed  first,  in  the  morning,  followed 
by  their  supervisors  and  subordinates  in  the  afternoon.  The  outcome 
could  not  have  been  better  planned.  Through  interviewing  all  of  the 
LMET  graduates  first,  the  interviewers  formed  a  crude  idea  of  the 
command's  attitude  towards  the  LMET  Program.  It  also  allowed  the 
interviewers  to  categorize  each  LMET  graduate  as  to  their  degree  of 
adoption  to  the  LMET  competencies.  Seldom  did  the  LMET  graduate 
interviews  exceed  the  allotted  60  minutes  which  allowed  the  inter¬ 
viewers  [active  and  passive]  to  quickly  exchange  impressions  on  each 
interview  while  still  fresh  in  their  minds. 
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The  most  beneficial  aspect  of  interviewing  the  LMET 
graduate  first  was  the  detailed  data  on  behavior  gained,  which  could 
later  be  validated  by  their  supervisor  and  subordinate.  In  a  few 
cases,  when  the  supervisor  or  subordinate  was  interviewed  first,  only 
general  responses  could  be  solicited  in  the  hope  that  they  would  in 
fact  substantiate  what  the  graduate  would  say.  The  interviewing 
process  operated  best  as  a  validation  of  behavior  when  conducted  in  the 
order  of  LMET  graduate,  then  either  subordinate  or  supervisor, 
c.  Interview  Environment 

All  interviews  for  this  pilot  study  were  conducted  either 
onboard  the  unit  or  working  spaces  and  areas  under  the  direct  control 
of  the  command.  This  kept  the  respondent  on  familiar  ground,  in  what 
could  be  termed  as  "home  territory".  However,  whenever  possible  a 
neutral  zone  or  area  was  used  in  order  to  remove  any  feelings  of  "big 
brother  watching  you".  For  enlisted  personnel  this  seemed  to  mean  any 
space  where  officers  did  not  have  ready  access  and  included  such  areas 
as  unoccupied  officer  staterooms,  unit  commander's  offices  [unit 
commander  not  embarked],  and  recreation  rooms  or  librarys.  In  one 
instance  the  wardroom  was  used  with  obvious  detremental  effects. 
Enlisted  respondents  appeared  to  be  uneasy  and  nervous,  and  stopped 
talking  to  look  and  see  who  entered  every  time  the  door  opened.  For 
officer  personnel,  any  place  appeared  to  be  sufficient.  In  remaining 
onboard  the  command  while  conducting  the  Interviews  it  appeared  to  be 
less  disruptive  and  more  ’Ikely  for  personnel  to  respond  promptly  to 
the  promulgated  time  schedule.  In  fact,  throughout  the  study,  inter¬ 
viewers  seldom  had  to  wait  more  than  a  few  minutes  and  respondents 


seldom  had  any  wait  at  all.  The  smoothness  with  which  the  schedule 
operated  was  in  no  small  part  responsible  for  the  large  number  of 
interviews  that  were  able  to  be  conducted. 

All  interviews  were  tape  recorded  instead  of  depending 
entirely  on  written  notes.  The  reason  for  the  use  of  the  tape  recorder 
and  the  confidentiality  of  the  interview  were  carefully  explained  to 
the  respondent  prior  to  each  interview.  After  obtaining  the  respond¬ 
ent's  permission  to  tape  the  interview  an  identification  statement  was 
made  on  the  tape.  The  statement  specifically  did  not  include  the 
respondent's  name  or  rate  to  reverify  the  confidentiality  that  had  been 
promised.  With  this  formality  over,  the  interview  proceeded  with 
little  or  no  attention  placed  on  the  tape  recorder.  It  did  not  appear 
to  inhibit  any  person  from  expressing  their  opinions.  The  tape  re¬ 
corder  could  be  slightly  disruptive  however,  if  a  tape  ended  in  the 
middle  of  an  interview.  The  respondent  would  normally  stop  talking 
until  a  new  tape  was  started  even  when  the  interviewer  insisted  that  he 
continue  and  not  worry  about  the  tape. 

Maintaining  the  neutrality  of  the  interviewing  environment 
is  extremely  important  as  was  demonstrated  during  the  course  of  two 
interviews  in  which  the  supervisor  was  nearby.  In  one  instance,  the 
supervisor  was  at  his  desk  on  the  other  side  of  an  equipment  rack,  and 
in  the  other,  he  was  actually  being  interviewed  simultaneously  across 
the  room.  In  both  instances,  the  respondent's  reactions  were  very 
similar.  The  respondents  appeared  to  be  nervous  and  continuously 
glanced  In  the  direction  of  their  supervisor.  One  respondent's  replies 
to  questions  seemed  to  be  for  the  benefit  of  his  supervisor.  The 
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respondents  seemed  to  realize  that  the  confidentiality  of  what  was 
being  said  could  not  be  guaranteed  under  these  circumstances, 
d.  Interviewer  -  Military  vs  Civilian 

Concerning  the  issue  of  military  versus  civilian  inter¬ 
viewers,  a  couple  of  points  are  of  note.  In  23  interview  sets  civilian 
clothes  instead  of  military  uniforms  were  worn  to  see  if  that  had  any 
effect  upon  responses.  It  appeared  to  the  interviews  that  the  wearing 
of  uniforms  was  optimal  over  civilian  clothes.  Instant  establishment 
of  credibility  as  a  person  knowledgeable  in  Navy  matters  exsisted  when 
wearing  a  uniform  that  seemed  to  increase  the  information  flow.  When 
civilian  clothes  were  worn  there  was  a  noticed  tendency  for  the 
respondent  to  explain  routine  procedures.  Navy  acronyms,  and  common 
programs,  that  were  not  done  when  interviewing  in  uniform.  In  both 
cases,  using  familiar  vocabulary  and  Navy  jargon  greatly  assisted  in 
conducting  the  interviewing  process.  Additionally,  a  feeling  of 
empathy  for  each  others  positon  was  quickly  established  when  in 
uniform.  The  rank  and  age  of  the  interviewers  also  seemed  to 
facilitate  the  entire  process.  Lieutenant  or  lieutenant  commander  is 
sufficiently  senior  to  develop  good  rapport  with  senior  petty  officers 
and  chief  petty  officers,  while  a  fairly  youthful  appearance  [age 
30-31,  one  interviewer  with  a  beard]  reduced  the  "generation  gap-  that 
might  have  inhibited  discussion  with  younger  officers  and  enlisted 
personnel.  The  fact  that  enlisted  personnel  are  quite  used  to 
interfacing  with  officers  but  not  with  civilians  in  an  official 
capacity  probably  had  a  positive  Impact  on  interviewing  in  uniform. 
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e.  Need  for  Interview  Introduction 

A  final  issue  that  surfaced  during  the  interviewing 
process  was  that  of  the  purpose.  Some  subordinates  seemed  reluctant  to 
talk  about  their  supervisor  despite  assurances  of  confidentiality.  At 
one  point  an  interview  was  briefly  stopped  and  it  was  explained  care¬ 
fully  to  the  respondent  that  the  intent  of  the  questions  and  study  was 
not  to  evaluate  his  supervisor's  performance  in  itself,  but  rather  to 
evaluate  a  program  of  leadership/management  techniques.  With  these 
assurances  the  interview  re summed  in  a  much  more  relaxed  and  co-opera¬ 
tive  atmosphere.  After  this  incident,  subordinate  interviews  were 
begun  by  including  this  performance  concept  as  part  of  the  overall 
interview  introduction  with  good  results. 

3.  Interview  Remarks 

The  interview  process  provided  an  excellent  method  for 
gathering  data  from  the  respondents.  Primarily  through  its  flexibil¬ 
ity,  the  researchers  were  afforded  the  opportunity  to  explore  relevant 
areas  of  interest  as  well  as  uncover  tangential  issues  of  concern. 
Concluding  interview  remarks  concerning  a  future  full-scale  evaluation 
are  contained  in  the  next  chapter.  The  remainder  of  this  chapter 
concerns  the  specific  LMET  content  information  derived  from  the 
interviews  themselves. 

8.  FINDINGS  CONCERNING  LMET 

The  findings  of  this  study  concerning  LMET  must  be  viewed  with 
full  recognition  of  the  limitations  of  the  sample.  The  sample  in¬ 
cluded  only  51  LMET  graduates  selected  from  13  operational  commands  in 


82 


three  west  coast  geographical  areas.  Furthermore,  the  sample  was  not 
random  and  therefore  cannot  be  said  or  alluded  to  be  truly  representa¬ 
tive  of  the  Navy.  However,  the  sample  did  contain  a  wide  variety  of 
warfare  specialties,  ranks,  rates  and  operational  commands,  and  does 
provide  a  good  insight  to  LMET's  influence,  particularly  on  the  sample 
population. 

The  method  of  describing  the  LMET  findings  which  will  be  addressed 
in  this  analysis  is  using  the  Kilpatrick  paradigm  consisting  of  four 
levels  or  domains  of  learning.  These  levels  are: 

1.  Reactions  [How  do  people  like  the  course?] 

2.  Knowledge-content  [What  did  they  learn?] 

3.  Behavior  [Has  their  behavior  changed?] 

4.  Results  [Does  it  make  any  difference  to  bottom  line 
measures?]  [McBER,1981] 

The  level  of  "reaction"  is  the  subjective  opinion  of  the  LMET  graduate 
towards  the  overall  course  usefulness.  This  data  can  be  gathered  at 
any  time,  but  is  probably  best  addressed  immediately  after  the  course 
[this  is  done  via  a  feedback  survey  completed  upon  completion  of  LMET 
training].  "Knowledge-content"  concerns  the  level  of  retaining 
information,  ideas  and  concepts.  It  also  is  best  measured  imnediately 
upon  completion  of  the  LMET  training  [which  is  occurring  in  response  to 
CNET  requests].  "Behavior"  is  the  level  in  which  the  instruction  is 
actually  practiced  on  the  job  by  the  LMET  graduate.  This  is  the  level 
in  which  the  pilot  study  interview  method  was  focused  upon  in 
attempting  to  measure  LMET  effectiveness.  The  final  area  of  "results" 
concerns  that  level  where  the  changed  behavior  of  the  LMET  graduate 
does  in  fact  bring  about  the  changes  within  the  system  which  are  de¬ 
sired.  This  is  in  the  realm  of  a  cause  and  effect  relationship  and 
amounts  to  a  prime  indicator  of  the  effectiveness  of  the  LMET  Program. 
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1.  Reactions 


The  first  of  the  four  levels  to  be  examined  is  that  of 
reactions.  While  it  was  not  the  intent  of  the  study  to  explore  this 
level,  responses  were  obtained  that  provide  some  insight  into  the  LMET 
graduate's  "reactions"  to  their  training, 
a.  LMET  Viewed  as  Necessary 

When  querried  as  to  whether  an  LMET  graduate  would  send 
his  personnel  to  the  school,  *he  responses  were  positive  [only  one 
negative  officer]  with  three  general  response  areas  given  most  often. 
First,  was  the  desire  to  increase  knowledge  in  the  area  of  leadership 
theory  and  to  increase  learning  and  vocabulary  in  the  field  of 
management.  Second,  was  the  idea  of  personal  improvement  through  a 
further  awareness,  either  to  "get-well"  or  to  make  good  leaders  better. 
Third,  is  that  individuals  are  required  to  go  through  LMET  school  via 
PCS  orders,  that  is,  it  is  an  expected  training  evolution  for  career 
personnel.  To  discuss  these  three  areas  in  more  detail,  the  data  [see 
Table  4]  indicates  that  an  overwhelming  majority  of  sampled  graduates 
view  LMET  school  as  a  useful  training  evolution.  Specifically,  the 
training  was  reported  to  provide  a  general  background  of  leadership/ 
management  theory  complete  with  a  specific  vocabulary.  This  new 
vocabulary  allows  the  graduate  to  be  able  to  converse  easier  with 
supervisors  and  subordinates  and  often  times  provides  names  and  labels 
to  leadership  styles  and  managerial  techniques  which  they  have  already 
been  practicing.  The  next  perception  held  by  graduates,  appears  to  be 
that  LMET  training  can  provide  poor  leaders  with  the  tools  necessary  to 
become  good  leaders  while  good  leaders  can  become  even  better.  Behind 
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TABLE  4 

REASONS  FOR  ATTENDING  LMET  SCHOOL 


Main  Interview  Reasons  Given  LMET  Graduate  Supervisor  Subordinate 

OFF/CPO/POI/Totals  (Totals)  (Totals) 


Acquire  management  techniques  5/ 

Learn  different  leadership 
styles  2/ 

Increase  "self-awareness"  4/ 

Better  leadership  skills 
or  to  "get  well"  1/ 

Learn  how  to  "get  along" 
with  people  0/ 

Learn  how  to  influence  people 
in  a  positive  manner  1/ 

Build  self-confidence  0/ 

Other*  2/ 


6/ 

5/ 

J6 

22 

_5 

3/ 

1/ 

_6 

_5 

_7_ 

0/ 

3/ 

_7 

_5 

22 

2/ 

1/ 

_4 

_2 

A 

0/ 

1/ 

_2 

_2 

0/ 

0/ 

A 

0/ 

1/ 

_1 

A 

A 

6/ 

6/ 

14 

mm 

mm 

Additional  Respondent  Comments: 


< 

i 


.  3  LMET  Graduates,  6  Supervisors  and  2  Subordinates  stated  that  some 
form  of  LMET  training  should  concentrate  on  E-4  and  above  instead  of 
E.6  and  above. 

-  Officer  LMET  Graduates  had  the  widest  range  of  remarks  as  seen  below: 
"....it  [LMET  School]  was  of  little  practical  value." 

" - a  ticket  punch." 

“ _ a  waste  of  time." 

" - can't  change  behavior." 

"....need  some  shipboard  experience  first." 

" - it  was  a  good  experience." 

" - prepared  me  for  a  leadership  role." 

"....I  now  understand  people  better." 

" - 1  learned  how  to  organize  my  time." 

" - it  was  good  background  enrichment." 

"....it  points  out  strengths  and  weaknesses." 


♦Comments  in  this  category  varied  and  were  generally  favorable  toward 
LMET  School.  Total  does  not  include  1  Commissioned  Officer  who  voiced  a 
negative  opinion.  Supervisor/Subordinate  columns  will  not  sum  to  51 
due  to  the  wide  variety  of  responses  received  and  not  included  in  the 
table. 
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this  notion  may  be  a  feeling  of  inadequacy  or  lack  of  time  in  which 
fleet  managers  and  leaders  feel  unable  to  provide  on-the-job  leader¬ 
ship/management  training.  The  final  perception  was  that  LMET  training 
is  a  promotion  hurdle  that  has  to  be  passed,  that  is,  the  training  is 
career  enhancing. 

b.  Why  LMET? 

In  addition  to  questioning  the  graduates,  supervisors  and 
subordinates  were  similarly  questioned  as  to  what  their  thoughts  were 
concerning  their  desire  to  attend  LMET.  Most  of  the  responses  from 
individuals  that  had  not  attended  the  LMET  training  concerned 

perceptions  about  what  LMET  could  do  for  them.  The  first  of  three 

general  perceptions  was  a  feeling  of  inadequacy  in  non-LMET  personnel 
[especially  junior  enlisted]  to  assume  positions  of  leadership  and 
management  without  formal  training.  Second,  was  the  feeling  of  needing 
additional  ideas  on  how  to  lead  and  manage  others,  that  is,  their 
actual  skills  needed  expanding.  The  third  impression  was  that  LMET 
training  would  provide  the  "right  way"  to  lead  and  manage  others. 

While  they  were  not  able  to  relate  these  ideas  to  the  LMET  school 
curriculum  itself,  it  does  imply  the  type  of  reputation  that  the 

program  possesses.  Fleet  personnel  look  upon  the  LMET  training  in  a 
positive  light.  They  expect  LMET  to  improve  their  leadership/manage¬ 
ment  skills  as  well  as  to  be  provided  with  the  "right  way"  to  lead  and 
manage  others.  In  summary,  the  reaction  to  LMET  training  seems  to 
receive  favorable  considerations  from  both  graduates  and  non-graduates 
alike.  As  stated  by  one  respondent,  "It  [LMET  training]  certainly 
can't  hurt!" 
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2.  Knowledge-Content 


The  second  level  or  domain  to  be  addressed  is  that  concerning 
what  the  LMET  graduates  actually  learned.  They  are  tested  on  this 
level  prior  to  leaving  the  school  and  the  interview  process  did  not 
specifically  seek  data  on  it,  nevertheless  several  insights  were 
revealed. 

a.  LMET  Knowledge 

For  those  graduates  who  had  been  away  from  the  school  for 
a  length  of  time  [6  months  or  longer],  there  were  indications  that  they 
had  forgotten  some  of  the  specific  content-knowledge  of  their  LMET 
training.  However,  as  the  interview  progressed,  they  tended  to  recall 
some  of  these  specifics  and  began  to  respond  to  interview  questions 
using  the  LMET  school  vocabulary.  The  actual  interviews  appeared  to 
surface  training  concepts  which  initially  appeared  to  have  been 
forgotten. 

b.  Setting  Goals  and  Performance  Standards 

Of  the  five  competency  clusters  taught  during  LMET  school, 
the  data  indicates  that  the  competency  group  of  efficiency  and 
effectiveness  seems  to  be  the  least  understood  and  practiced.  Concern¬ 
ing  efficiency  and  effectiveness,  it  can  be  paraphrased  as,  "Doing 
things  well,  and  wanting  to  do  better".  [HRMC,N.D.]  It  includes 
setting  goals  and  performance  standards. 

8ehaviors  demonstrated  by  one  who  sets  goals 

and  performance  standards  include  the  following: 
♦Establishes  specific  work  goals 
♦Is  concerned  for  standards  of  task  performance 
♦Revises  goals  to  make  them  realistic 
♦Sets  deadlines  for  task  accomplishment  [HRMC.N.D.] 


As  indicated  in  Table  5  and  6,  LMET  graduates  expressed  adequate 
knowledge  concerning  goal  setting,  establishing  performance  standards 
for  subordinates  and  clarifing  evaluation  standards.  However,  when 
specifically  asked  whether  they  infact  practiced  these  behaviors  the 
response  was  negative  as  also  validated  by  their  subordinates.  The 
implication  is  that  this  competency  cluster  remains  at  the 
knowledge-content  domain.  It  would  appear  from  the  sample  that  LMET 
graduates  do  not  routinely  practice  setting  goals  and  performance 
standards  for  their  subordinates.  Additionally,  LMET  graduates  sampled 
do  not  seek  clarification  from  their  supervisors  as  to  goals  and 
expectations  that  apply  to  themselves.  As  best  expressed  by  an  LMET 

petty  officer  graduate  " - 1  guess  I  don't  really  tell  them  what  I 

expect... I  know  I  should." 

In  sunmary,  the  data  analyzed  from  the  interviews  within 
the  13  commands  indicates  that  sampled  LMET  graduates  are  knowledgeable 
concerning  the  content  of  their  training.  However,  there  appears  to  be 
a  lack  of  application  in  certain  areas.  A  lack  of  application  that 
some  freely  admit. 

3.  Behavior 

The  pilot  study  research  was  directed  specifically  at  this 
third  domain  -  the  behavioral  level,  that  is,  whether  the  LMET  gradu¬ 
ates  exhibit  any  behavioral  changes.  Overall,  there  were  no  systematic 
behavioral  changes  that  ran  across  the  sample.  However,  there  were 
isolated  behavioral  changes  which  were  clearly  the  result  of  LMET 
training.  For  those  several  cases  where  a  sharp  and  drastic  change 
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TABLE  5 

GOAL  SETTING  BY  LMET  GRADUATES 


The  question  asked  to  LMET  graduates  was,  "What  goals  did  you  set  in 
LMET  school?  Follow-up  questioning  and  probing  attempted  to  determine  if 
goal  setting  was  used  routinely  as  a  management  tool. 


Response  to  Initial  Question 

Officer 

CPO 

P01 

1.  Measurable  goal  dealing  with 
immediate  future  leadership/ 
management  position 

4 

7 

8 

2.  Personal  goal 

2 

5 

5 

3.  Vague  goal  involving  some 
phase  of  leadership  [i.e. 
"achieve  good  leadership", 
"improve  leadership  style", 
"become  a  better  manager"] 

2 

4 

3 

4.  Forgot/don't  remember 

J3 

J_ 

_2 

TOTALS 

16 

17 

18 

Follow-up  question  results: 

Officer 

CPO 

P01 

Number  of  LMET  graduates  who  have 
established  work  group  goals[X] 

9/16  [56] 

16/17  [94] 

16/18 

[89] 

Number  of  LMET  graduates  setting 
measurable  goals[%] 

4/16  [25] 

8/17  [47] 

9/18 

[50] 

Number  of  LMET  graduates  setting 
goals  with  time  frames[%] 

0/16  [0] 

3/17  [18] 

6/18 

[33] 

TABLE  6 

SETTING  PERFORMANCE  STANDARDS  BY  LMET  GRADUATES 

The  question  concerning  performance  measures  was,  "How  is  your  per¬ 
formance  on  the  job  measured?"  Additional  probing  was  aimed  at  estab¬ 
lishing  specific  measures.  A  rephrasing  of  the  question  to,  "How  do  you 
know  when  you're  doing  a  good  job?",  was  particularly  effective. 

I.  LMET  Graduate  Self-Report  Response* 


Performance  Measurements 

OFF 

CPO 

P01 

Task  accomplishment  ["get  the  job  done"] 

13 

13 

14 

Technical  proficiency  in  rate  or  specialty 

4 

6 

3 

Manage  the  work  group  [emphasis  on  smooth¬ 
ness  of  operation] 

1 

5 

3 

Do  not  know 

2 

2 

4 

Amount  of  feedback  provided  supervisor 

4 

4 

1 

Effective  training  program 

1 

2 

1 

Establishing  good  inter-personal  re¬ 
lationship  with  subordinates 

2 

0 

1 

Goal  setting 

2 

0 

0 

Prioritizing  workload  properly 

2 

3 

0 

Keep  supervisor  out  of  trouble 

1 

1 

2 

Appearance  of  workgroup  personnel 

1 

1 

1 

Material  Resource  Management 

1 

1 

1 

Morale  of  work  group  personnel 

2 

1 

0 

Keeping  up  with  administration  [paper  work] 

2 

0 

0 

Amount  of  flexibility 

1 

0 

2 

Monitoring  work  group 

0 

2 

1 

Work  group  planning 

0 

3 

1 

*Col turns  may  total  to  more  than  the  nunber  of  graduates  due  to  multiple 
responses. 
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TABLE  6  [con't] 


II.  Validated  LMET  Graduate  Performance  Measurements* 


Supervisor 

Validation 

LMET 

Agree  with 
graduate 

Disagree  with 
graduate 

graduates 
not  knowinq 

Officer 

9 

3 

2 

CPO 

10 

3 

2 

P01 

9 

4 

4 

Totals: 

28 

10 

8 

III.  PERCEIVED 

PERFORMANCE  STANDARDS 

Do  not  know 

OF  SUBORDINATES  TO  LMET 

Task  Accomplishment 

GRAUDATES 

Others 

Officer 

4 

7 

1 

CPO 

4 

9 

4 

P01 

3 

12 

2 

Totals: 

11 

28 

7 

♦This  tabulation  presents  the  LKET  graduates  perceived  performance 
measurements  as  opposed  to  the  immediate  supervisor's  stated  performance 
measurements.  Totals  may  not  equal  number  of  graduates  due  to  supervisor 
or  subordinate  not  interviewed  for  this  question. 


took  place  the  personnel  were  clearly  looking  for  help.^  They 
attended  LMET  school  as  a  "get  well"  measure  with  a  substantial  felt 
need  to  improve.  In  general,  those  that  exhibited  the  hoped  for 
behaviors  or  competencies  indicated  that  they  did  so  prior  to  LMET 
school,  while  those  not  exhibiting  the  behaviors  desired  did  not  see 
the  value  of  those  competencies,  feeling  they  were  not  applicable  in 
their  specific  situation. 

a.  Seniority  among  LMET  Graduates 

The  data  seemed  to  indicate  that  there  may  be  a 
correlation  [negative]  between  senority  and  extent  of  behavioral 
change.  As  Table  7  seems  to  Indicate,  chief  petty  officers  are  less 
affected  by  LMET  training  as  far  as  changing  their  behavior.  While 


there  is  some  subjective  indication  that  they  in  fact  can  better 
appreciate  what  is  being  taught  and  how  it  applies  due  to  their 


experience,  it  seems  that  they  tend  to  be  more  skeptical  towards  new 

ideas.  The  tendency  seems  to  be,  to  use  the  LMET  training  as  a 

reinforcement  of  their  own  leadership/managerial  style  by  adhering  to 

those  competencies  that  reflect  their  own  style.  The  notion  of  more 

senior  LMET  graduates  using  LMET  to  reinforce  past  behavior,  thereby 

reducing  the  likelihood  of  behavioral  change  is  reflected  in  the 

following  chief  petty  officer's  comments: 

...not  really,  [no  change]  I  found  that  what  they 
brought  out  in  LMET  was  basically  what  I  was  doing 
before  I  went. 

...you  might  be  able  to  modify  a  man  but  you  can't 
change  his  behavior. 

...I  don't  think  so. . .[behavioral  change]  there  are 
some  things  they  pointed  out  that  I  do  use... the  way 
I  handle  the  men  is  the  same. 


was  validated  by  supervisors  and 


subordinates  who  observed  the  subjects  before  and  after  LMET  school. 
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TABLE  7 

EXHIBITED  BEHAVIORAL  CHANGE  OF  LMET  GRADUATES 


Yes  Yes  Yes/No*  No  No 

[A  great  deal  [Minimum  [Disagreement  [little  [Definatly 

of  change]  Change]  between  graduate  change  None] 

&  supervisor/  if  any] 
subordinate] 


Officer 

2 

5 

2/ 

3 

4 

CPO 

3 

2 

2/0 

0 

10 

P01 

6 

1 

7/1 

0 

3 

Total 

JJC21.6X] 

8[15.7X] 

11/1[21.6X/1.9X]  3[5.9%] 

V7[33.3X] 

J9[37.3X] 

22[23.5X] 

20[39.2X] 

*In  this  column  ''YesM  means  the  graduate  states  there  has  been  a  change  and 
supervisor/subordinate  state  no  change.  A  "No"  means  the  graduate  believes 
there  has  been  no  change  while  supervisor  and  subordinate  say  there  has  been 
a  behavioral  change. 


I  can't  say  that  [leadership  style]  has  really 
changed. . .basically  I  am  the  same  person,  the  way 
I  lead  my  people. 

No,  I  think  LMET  made  me  realize  some  of  my 
strengths  and  weaknesses  so  far  as  leading  men, 
however,  I  thought  I  had  more  strengths  than 
weaknesses. 

The  more  junior  officers  and  petty  officers  seem  to  view 
LMET  as  a  valuable  and  much  needed  training  program.  Among  first  class 
petty  officers  in  particular,  LMET  training  seems  to  provide  the  leader¬ 
ship/management  theories  and  tools  with  which  they  are  able  to  do  their 
jobs.  Additionally,  this  group  has  had  sufficient  experience  to 
appreciate  the  curriculum  structure  and  content.  The  data  seems  to 
indicate  that  first  class  petty  officers  are  the  most  affected  group  of 
the  3  sampled  levels  and  exhibit  the  greatest  potential  for  behavioral 
change.  For  sampled  junior  officers,  especially  ensigns,  there  seems  to 
be  less  indication  of  any  behavioral  change  [for  new  ensigns  this  is 
primarily  due  to  a  lack  of  experience  in  not  knowing  how  they  led  and 
managed  others  prior  to  LMET].  While  these  junior  officers  express  a 
great  felt  need  for  the  LMET  training  with  its  role  in  preparing  them  to 
assume  their  duties,  their  behavior  does  not  indicate  that  they  readily 
apply  the  competencies.  Perhaps  as  was  suggested  by  the  junior  officers 
themselves,  LMET  school  [as  presently  scheduled],  sandwiched  between 
Surface  Warfare  Officer  School  and  other  specific  training  schools 
[Communication  Officer,  OCA  School,  Electronic  Warfare,  etc.],  looses 
its  impact  by  the  time  the  officers  reach  the  fleet.  Then  in  the  rush 
to  fit  into  their  operational  command  and  specific  billet  the  LMET 
training  seems  to  get  lost  and  forgotten,  perhaps  due  to  their  inexperi¬ 
ence  causing  a  lack  of  appreciation  for  the  course  content. 
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b.  Weak  Communications 


Of  the  five  competency  clusters  taught  during  LMET  school, 
two  are  particularly  dependent  upon  good  communication  flow.  The  first 
cluster  concerns  the  skillful  use  of  influence  and  is  paraphrased  as: 
"Using  influence  in  a  positive  fashion. . .not  as  a  personal  end-but 
towards  Navy  goals  and  effectiveness."  [HRMC.N.D.]  Components  of  this 
competency  cluster  include,  influences,  team  builds,  developes  sub¬ 
ordinates  and  self-control.  The  thread  that  runs  throughout  this 
competency  is  communication.  The  analysis  of  data  gathered  during  the 
study  [Table  8]  seems  to  indicate  that  sampled  LMET  graduates  do  not 
apply  this  competency  to  its  fullest  extent  due  to  poor  communications. 

A  lack  of  communication,  specifically  in  feedback  concern¬ 
ing  performance,  was  sited  throughout  the  interviews  of  both  graduates 
and  their  subordinates  and  supervisors.  Interestingly,  it  included  both 
the  lack  of  positive  as  well  as  negative  feedback,  but  especially 
emphasized  the  notion  of  positive  feedback.  This  lack  of  positive  feed¬ 
back  among  those  interviewed  indicated  that  a  substitute  for  positive 
feedback  became  the  very  lack  of  any  feedback.  When  querried  as  to  how 
they  knew  when  they  were  doing  a  good  job,  a  common  response  from 
sampled  graduates  and  subordinates  was:  "I  really  don't  know,  if  I  don't 
hear  anything  then  I  sust  be  doing  all  right."  This  substitution  of  a 
lack  of  any  communication  for  actual  positive  feedback  appeared  to  be  a 
common  occurence  among  respondents. 

Other  indications  of  poor  communication  involved  a  lack  of 
including  subordinates  in  the  planning  stages  of  task  accomplishment 
[Table  8].  When  the  subordinate's  opinions  were  solicited  they  were 
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TABLE  8 

LMET  GRADUATE  SELF-REPORT  ON  INFLUENCING 


This  data  was  gathered  during  the  course  of  answering  three  related 
questions  concerning  the  process  used  to  accomplish  a  newly  assigned 
task,  assigning  duties  and  responsibilities  to  subordinates  and  the 
handling  of  roadblocks.  Respondents  were  encouraged  to  site  specific 
examples.  The  analysis  rated  the  answers  on  a  scale  of  1-5  within  the 
four  relevant  competency  clusters. 


Officer  LMET  Graduates 


Low  or  No  Use 
of  LMET  Compentency 

1  2 

3 

High  Use  of  Lf 
Competency 
4  5 

Efficiency  & 
Effectiveness 

0  7 

4 

8 

3 

Skillful  Use  of 

Influence 

1  2 

5 

4 

4 

Management  Control 

0  0 

2 

6 

8 

Conceptual  Thinking 

0  1 

3 

3 

9 

OVERALL: 

1  4 

14 

21 

24 
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TABLE  8  [con't] 


Low  use  of 

Chief  Petty  Officer  LMET  Competency 

LMt'T  Graduate  1 

2 

3 

High  use  of 
LMET  Competency 
4  5 

Efficiency  &  Effectiveness 

0 

0 

2 

3  12 

Skillful  use  of  Influence 

0 

2 

3 

5  7 

Management  Control 

0 

2 

0 

5  10 

Conceptual  Thinking 

0 

2 

0 

6  9 

OVERALL: 

0 

6 

5 

19  38 

First  Class  Petty  Officer 
LMET  Graduate 

1 

2 

3 

4  5 

Efficiency  &  Effectiveness 

0 

1 

2 

6  9 

Skillful  use  of  Influence 

1 

4 

2 

4  7 

Management  Control 

0 

1 

2 

3  12 

Conceptual  Thinking 

0 

2 

3 

5  9 

OVERALL: 

1 

8 

9 

18  36 

Additional  data  on  the  "Skillful  Use  of  Influence"  competency  cluster 
was  gained  by  asking  the  graduate,  "How  do  you  go  about  getting  someone  to 
buy  your  ideas  about  a  better  way  of  doing  things?" 

Officer 

Chief  Petty 
Officer 

First  Class 
Petty  Officer 

TOTAL 

Do  nothing  unless  asked 

0 

0 

2 

2 

Order  it  done 

3 

1 

2 

6 

Argue  that  is  the  way  it  has 
always  been  done  or  you  have 
seen  it  done  this  way  before 
If  accepted  fine. 

1 

7 

4 

12 

Trial  and  Error  Approach 

2 

0 

4 

6 

Argue  strongly  in  its  favor, 
pointing  out  strengths,  make 
stand  on  its  own  merit. 

it 

10 

9 

6 

25 
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often  rejected  with  little  to  no  explanation,  or  with  out  trying  them 
out.  A  common  practice  seemed  to  be  not  consulting  with  subordinates  at 
all. 

"Advising  and  Counseling"  is  the  second  competency  cluster 
affected  by  a  weakness  in  communications  and  includes  the  components; 
positive  expectations,  realistic  expectations  and  understanding.  This 
competency  group  places  particular  emphasis  on  the  communication  aspect 
under  the  component  of  expectations.  Realistic  expections  include: 

*A  realistic  concern  that  instructions  will  be 
followed  or  carried  out  effectively  by  others. 

♦Open  acknowledgement  of  negative  expectations 
about  the  shortcomings  of  others. 

*A  willingness  to  express  displeasure,  dis¬ 
appointment,  and  concern  about  the  short¬ 
comings  of  an  individual's  performance. 

[HRMO,N.O.] 

Again,  communication  is  seen  as  an  integral  part  of  this  competency. 

The  interviews  seem  to  indicate  that  in  this  area  there  is  a  great  deal 
of  communication  between  graduates  and  their  subordinates.  However,  the 
emphasis  appears  to  be  only  on  the  negative  aspect  while  ignoring  the 
other  components  of  positive  expectations  and  understanding.  As  Table  9 
indicates,  supervisors  are  quick  to  provide  verbal  feedback  and 
counseling  for  poor  performance.  Yet  for  good  performance,  none  of  the 
sampled  LMET  graduates  indicated  they  would  specifically  counsel  their 
subordinates.  However,  verbal  feedback  in  the  form  of  a  "job  well  done" 
was  given.  The  indication  seems  to  be  that  the  sampled  LMET  graduates 
view  counseling  as  a  tool  for  correcting  poor  performance  with  much  less 
emphasis  on  good  performance. 
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c.  Rewarding  and  Disciplining  Performance 

The  fourth  competency  cluster  taught  during  LMET  school  is 
that  of  "Management  Control",  which  includes  the  following  six 
components: 

♦Plans  and  Organizes 
♦Optimizes  use  of  resources 
♦Delegates 
♦Monitors  results 
♦Rewards 

♦Disciplines  [HRMC,N.D.] 

The  first  four  components  of  this  competency  appears  to  be 

used  with  varying  degrees  of  success  as  indicated  by  the  data  [Table  8]. 

While  sampled  LMET  graduates  did  not  always  indicate  they  involved 

others  in  the  planning,  they  certainly  made  efforts  to  optimize 

resources  and  delegate  tasks  to  a  lower  level.  The  last  two  components, 

rewards  and  disciplines,  did  not  appear  to  be  integrated  into  the 

sampled  LMET  graduates  behavior. 

As  indicated  in  Table  9,  the  use  of  rewards  was 

substantially  limited  to  three  areas,  verbal  recognition  [formal  and 

informal],  time  off  from  work,  and  written  evaluations.  Of  these  three 

responses,  written  evaluations  may  be  downplayed,  as  few  respondents 

suggested  special  or  commendatory  evaluations  but  rather  indicated  good 

performance  would  be  noted  on  formal,  periodic  written  evaluations. 

Since  the  Navy  evaluation  process  expects  or  insists  supervisors  do 

this,  it  cannot  be  truly  viewed  as  much  as  a  reward  as  something  earned. 

This  feeling  was  best  expressed  by  LMET  subordinates,  one  who  remarked, 

"He  didn't  give  me  a  good  evaluation,  I  earned  it!"  The  other  two 

major  types  of  rewards  were  verbal  rewards  and  time  off  from  work,  with 

both  more  heavily  relied  upon  by  the  first  class  petty  officers. 

Again,  these  rewards  have  come  to  be  so 
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TABLE  9 

LMET  GRADUATES  SELF-REPORT  ON  ADVISING  AND  COUNSELING 
AND  REWARDS  AND  DISCIPLINES 


This  data  was  gathered  from  responses  to  two  related  questons, 
"What  do  you  do  if  the  work  of  a  subordinate  is:  worse  than  expected? 


Better  than  expected?"* 

No  or  Low  use 
of  LMET 
Competencies 

Officer  1  2 

3 

4 

High  use  of 
LMET  Competencies 
5 

Efficiency  &  Effectiveness 

1 

1 

3 

5 

6 

Skillful  use  of  Influence 

0 

2 

6 

2 

6 

Advising  and  Counseling 

3 

0 

1 

3 

9 

Management  Control 

0 

3 

3 

3 

7 

OVERALL: 

4 

6 

13 

13 

28 

Chief  Petty  Officer  _ 

1 

2 

3 

4 

5 

Efficiency  &  Effectiveness 

0 

0 

1 

8 

8 

Skillful  use  of  Influence 

0 

1 

5 

5 

6 

Advising  and  Counseling 

0 

1 

1 

6 

9 

Management  Control 

0 

1 

4 

7 

5 

OVERALL: 

0 

3 

n 

26 

28 

*This  Table  was  prepared  using  the  LMET  graduate's  responses  from  the 
interview  tapes.  The  scale  was  developed  as  a  convenient  method  of 
presenting  the  data. 


TABLE 

9  [con't] 

First  Class  Petty  Officer 

1 

2  3 

4 

5 

Efficiency  &  Effectiveness 

0 

0  1 

8 

9 

Skillful  use  of 

Influence 

0 

1  7 

3 

7 

Advising  and  Counseling 

1 

1  0 

7 

9 

Management  Control 

0 

1  6 

5 

6 

OVERALL: 

1 

3  14 

23 

31 

Specific  responses  to  the  two  querries  follow: 

"If  work  is  worse" 

• 

Counsel 

Verbal 

Reprimand 

Re-Do  Extra-Military  Formal  Reassign 

Task  Instructional]  Punishment  Task 

Officer  13 

8 

3 

2 

3 

2 

CPO  15 

12 

6 

0 

3 

0 

r  , 

S 

P01  15 

10 

5 

5 

4 

0 

"If  work  is  better" 

( 

Verbal 

Reward 

Physical /Material 
Award  [Buy  a  coke, 
handshake,  pat  on 
back,  etc.] 

Time-Off 

Written 

Evaluation  Formal  Award 

Recommend a-  Commendation 

tion  for 

advancement 

% 

Officer  11 

4 

10 

6 

6 

CPO  15 

4 

10 

9 

8 

■  < 

t  ’ 

P01  16 

6 

16 

11 

4 

i 
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institutionalized  as  to  almost  lose  their  status  as  rewards.  Personnel 
who  work  late,  especially  hard  or  do  a  job  particularly  well  expect  to 
be  rewarded  both  verbally  and  with  time  off.  To  not  give  these  rewards 
seems  to  be  viewed  as  a  form  of  punishment.  In  addition,  all  three  of 
these  rewards  require  only  a  small  amount  of  effort  for  the  supervisor, 
hence,  the  propensity  towards  their  use.  The  fact  that  chief  petty 
officers  and  officers  are  reluctant  to  give  time  off  as  compared  to 
petty  officers  may  be  indicative  of  their  view  of  the  organization,  the 
amount  of  work  to  be  completed  as  well  as  more  direct  pressure  from 
seniors  to  account  for  absent  personnel.  Written  awards,  commendations 
and  recommendations  for  medals  was  given  a  lower  priority  as  a  means  of 
rewarding  personnel  for  good  performance  for  a  variety  of  reasons.  Most 
common  seemed  to  be  the  concept  that  good  performance  was  expected, 
hence  not  worthy  of  such  a  "high"  reward.  Yet  few  graduates  were  able 
to  verbalize  what  sort  of  performance  did  deserve  such  an  award.  It  was 
the  interviewer's  perception  that  the  problem  was  not  the  level  of  the 
award  but  the  amount  of  personal  involvement  needed  to  give  such  an 
award. 

The  last  component,  discipline,  seemed  to  share  the  same 
sort  of  priorities  as  rewards.  That  is,  disciplinary  actions  that 
involved  a  substantial  amount  of  interaction  with  the  supervisor  were 
avoided.  As  Table  9  indicates,  counseling  [negative  feedback]  and 
verbal  reprimands  were  the  most  commonly  cited  means  of  disciplining. 
It  Is  perhaps  no  coincidence  that  these  forms  involved  the  least  amount 
of  a  supervisor's  effort.  Even  "counseling"  can  be  conducted  with 
little  preparation  and  informally  so  as  to  be  very  efficient  from  a  time 
perspective.  Redoing  the  task  and  Extra  Military  Instruction  [EMI]  both 


involve  a  high  degree  of  interaction  with  the  supervisor  in  the  form  of 
close  monitoring  and  instruction.  Both  methods  were  shunned  as 
Indicated  by  the  data,  especially  at  the  officer/chief  petty  officer 
level.  Actual  punitive  measures  such  as  non-judicial  Captain's  Mast 
were  viewed  by  those  respondents  who  mentioned  it  as  a  last  ditch 
effort,  only  to  be  used  when  all  else  failed.  What  is  significant  is 
the  relatively  few  respondents  that  even  mentioned  it.  A  final  note 
should  be  made  concerning  the  officers  who  would  reassign  a  task 
improperly  completed  to  another  person  who  could  be  relied  upon  to  get 
it  completed.  This  would  seem  to  indicate  an  extreme  propensity  for 
task  accomplishment  at  any  cost,  and  if  so,  would  be  highly  divergent  to 
the  LMET  approach  to  leadership. 

d.  Increased  Awareness 

A  final  area  of  behavioral  change  concerns  overall  change, 
perhaps  better  called  an  awareness  or  attitudinal  change.  That  is,  the 
graduate's  awareness  towards,  leadership/management  as  well  as  his 
affect  on  other  people  has  increased  which  is  manifested  in  some 
behavioral  changes.  Table  7  indicates  that  approximately  two  thirds 
[6156]  of  the  sampled  graduates  expressed  either  a  change  in  their 
leadership  style  or  in  adopting  a  leadership  philosophy  more  closely  in 
line  with  LMET.  Of  these  graduates,  half  have  in  fact  changed  or 
adopted  an  LMET  approach  as  validated  by  their  supervisors  and 
subordinates.  One  third  of  the  total  sample  have  exhibited  no  change  as 
validated  by  their  supervisors  and  subordinates.  This  means  about  one 
fourth  [23.556]  of  all  graduates  believe  they  have  changed  yet  this 
change  is  not  validated.  The  significance  of  this  data  seems  to  be  that 

behavioral  change  brought  about  by  LMET  varies  greatly,  based  upon  a 
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variety  of  factors.  However,  it  does  seem  clear  that  behavioral  change 
as  a  specific  result  of  LMET  school  is  not  at  all  assured. 

4.  Results 

The  last  domain  or  level  to  be  examined  is  that  of  results, 
which  in  fact  is  what  an  evaluation  of  the  entire  LMET  Program  should 
most  likely  explore.  The  priority  question  is  whether  the  program  does 
in  fact  result  in  real  changes  to  the  bottom  line  measures.  This  pilot 
study  was  not  designed  to  definitively  address  the  specific  domain  of 
"results".  Without  a  control  group  of  non-LMET  graduates  to  compare 
against,  and  the  numerous  non-LMET  related  factors  that  affect 
productivity  [e.g.,  technical  training,  size  of  work  group,  imagination, 
etc.]  there  may  be  no  satisfactory  method  of  determining  the  cause  and 
effect  relationship  between  LMET  and  "results"  outcomes.  Despite  these 
difficulties  and  recognizing  the  limitations  of  the  sample  itself,  some 
indications  of  the  effect  of  LMET  on  bottom  line  measurements  were 
uncovered. 

a.  Job  Performance  Changes 

First,  most  sampled  LMET  graduates  reported  that  job  per¬ 
formance  measures  had  increased  or  remained  high  in  their  work  groups 
[this  was  validated  by  both  subordinates  and  supervisors]  since  their 
completion  of  training  [see  Table  10].  Of  particular  interest  is  that 
first  class  petty  officers  report  lower  perceived  impact  on  work  group 
performance  than  do  the  chief  petty  officers  or  officers.  This  may 
reflect  the  first  class  petty  officer's  perception  that  being  lower  in 
the  chain  of  command  affords  them  less  chance  to  influence  the  system. 
The  relatively  high  percentage  of  officers  [63<]  that  saw  an  Increase 
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TABLE  10 


CHANGE  IN  THE  LMET  GRADUATE'S  WORK  GROUP  PERFORMANCE  AND  SATISFACTION 


Perceived  Performance  Changes: 


Yes, 

Yes, 

No, 

No 

No, 

Yes, 

Yes, 

major 

minimum 

remained 

remained 

Minor 

major 

increase 

increase 

high 

low 

decrease 

decrease 

Officer  6 

4 

1 

3 

0 

1 

1 

CPO  4 

7 

3 

3 

0 

0 

0 

P01  4 

6 

1 

6 

1 

0 

0 

*— 

“ * 

OVERALL:  14  17  5 

Perceived  Satisfaction  Changes 

12 

1 

1 

1 

Officer  7 

2 

0 

5 

0 

1 

1 

CPO  1 

7 

4 

2 

3 

0 

0 

P01  3 

7 

1 

6 

l 

0 

0 

* 

OVERALL:  11 

16 

5 

13 

4 

1 

1 

may  well  reflect  a  willingness  on  their  part  to  experiment  with 
leadership/managerial  styles  as  well  as  use  of  their  relative  position 
of  higher  influence.  Chief  petty  officers  may  also  enjoy  a  position 

form  which  it  is  easier  to  influence  improved  performance.  This 
hypothesis  is  supported  by  a  comment  from  one  subordinate  who  stated, 
"We  got  a  chief  so  things  got  better." 
b.  Job  Satisfaction  Changes 

As  for  satisfaction  measures  within  a  work  group  [increased 
retention,  less  non-judicial  punishments,  less  sick  calls  and 
absenteeism],  there  was  no  consistent  pattern  of  change  across  the 
entire  sample  which  could  be  attributed  to  LMET.  While  Table  10  seems 
to  indicate  some  positive  changes  in  satisfation,  the  respondents 
credited  most  change  to  environmental  changes  [e.g.,  return  from 
deployment,  finishing  shipyard  overhaul,  etc.]  or  to  the  personality 
[e.g.,  nice  person,  good  conversationist,  easy  going,  etc.]  of  the  LMET 
graduate  and  not  so  much  to  the  leadership  style.  However,  the  inter¬ 
views  seemed  to  indicate  that  when  LMET  graduates  did  infact  involve 
subordinates  in  the  decision  making  process  and  opened  clear  lines  of 
communication  that  job  satisfaction  did  increase.  Of  particular  note 
was  the  case  of  two  petty  officers  who  did  a  complete  turnaround  in 
leadership/management  styles  after  LMET  [attended  on  a  TAD  basis].  In 
their  cases  the  change  was  clearly  noticable  to  supervisors  and 
subordinates  and  in  fact  resulted  in  measureable  increases  in  both  job 
performance  and  satisfaction  measures. 


c.  Retention  Aspects  of  LMET 

Concerning  retention,  the  data  do  not  indicate  any 

systematic  cause  and  effect  relationships  between  retention  and  LMET 
training.  However,  there  were  specific  cases  in  which  there  appeared  to 
be  an  LMET  influence  toward  increased  retention  within  the  work  group. 
The  graduates  themselves  indicated  that  LMET  training  had  little  effect 
on  their  own  career  plans.  This  is  perhaps  not  surprising  when 
examining  who  is  actually  attending  LMET  school.  Chief  petty  officers 
and  first  class  petty  officers  tend  to  be  career  oriented  personnel  or 
have  at  least  made  definate  plans  concerning  re-enlistment.  Regardless, 
they  tend  to  be  a  tough  group  to  sway  one  direction  or  the  other. 

Junior  officers  on  the  other  hand,  while  more  undecided  about  their 
careers,  appear  to-be  unwilling  to  make  a  career  committment  based  soley 
upon  a  positive  LMET  experience. 

LMET  graduates  on  the  other  .hand,  did  have  some  positive 
effect  on  retention  within  their  work  groups.  Several  subordinates 
readily  admitted  that  the  actions  of  their  LMET  graduate  boss  had  in 
fact  been  the  determining  factor  in  their  re-enlistment  decision.  One 
LMET  graduate  had  a  100  per  cent  re-enlistment  rate  [supervisor-report¬ 
ed].  However,  the  question  still  remains  as  to  whether  this  was  all  or 
even  in  part  due  to  LMET.  At  least  one  certainty  can  be  stated,  that  is 

that  no  respondent  indicated  they  were  leaving  the  Navy  as  a  result  of 

LMET. 

d.  Developing  Healthy  Climates 

With  respect  to  developing  healthy  organizations  or  work 
groups,  LMET  graduates  seemed  to  give  this  a  high  priority  and  are  in 
fact  successful  as  indicated  In  Table  11.  The  majority  of  the  sampled 
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TABLE  11 


INDICATION  OF  HEALTHY  WORK  GROUPS  OF  LMET  GRADUATES 


This  data  was  gathered  on  the  question,  “What  happens  to  your  work 


center  when 

you  are  not 

there?- 

1 

Things  fall 
apart 

2 

Just  enough 
Is  done  to 
stay  out  of 
trouble 

3 

Major  jobs 
continue, 
some  minor 
ones  don't 

4 

All  work 
done,  accept 
supervisors 

5 

All  work 
gets  done 
including 
supervisor 

Officer* 

1 

1 

4 

7 

2 

CPO 

1 

1 

7 

3** 

5 

POl*** 

l 

0 

4 

3 

9 

OVERALL: 

3 

2 

15 

13 

16 

♦One  officer  was  not  asked  this  question  as  he  was  in  the  process  of 
relieving  the  old  division  officer. 

♦♦These  chief  petty  officers  all  specified  that  only  the  division  officers 
collateral  duties  would  not  get  done. 

♦♦♦One  POl  was  not  asked  this  question  due  to  his  newness  to  the  work  group. 


LMET  graduates  stated  that  they  had  in  fact  trained  their  subordinates 
to  take  over  in  their  absence.  There  was  some  question  as  to  how  well 
their  work  group  would  function  [quality  of  performance],  but  not 
whether  it  would  function.  In  those  few  instances  when  the  LMET 
graduate  indicated  that  the  work  group  would  not  function  at  all  or  at  a 
level  to  barely  get  by,  the  LMET  graduate  appeared  to  not  practice  the 
LMET  competencies,  especially  skillful  use  of  influence.  The  overall 
indications  were  that  LMET  graduates  who  practiced  what  was  taught, 
tended  to  create  healthy  organizations  by  training  subordinates  to  take 
on  increased  responsibility,  delegating  decisions  to  lowest  possible 
levels,  encouraging  communication  up  and  down  the  chain  of  command  and 
using  their  influence  to  positively  motivate  their  subordinates, 
e.  Handling  Newly  Reported  Personnel 

A  related  area  to  retention  and  building  a  healthy  work 
group  is  how  new  personnel  are  integrated  into  the  work  group.  Table  12 
indicates  that  LMET  graduates  tend  to  concern  themselves  with  a  new 
persons  first  impression.  Most  graduates  stressed  the  use  of  the 
reporting  aboard  interview  as  a  means  of  explaining  where  the  new  person 
fits  within  the  organization  as  well  as  his  importance  to  the 
organization.  The  interview  was  also  used  to  inform  the  new  individual 
of  promotion  plans,  career  development,  training  programs,  and  daily 
routines/standard  operating  procedures.  This  interview  stressed  the 
necessity  of  discussing  any  personal  problems  that  might  prevent  easy 
transition  into  the  work  group.  The  one  subject  that  was  not  discussed 
tended  to  be  the  expected  performance  or  goals  for  the  newly  reported 
person.  Most  respondents  stated  it  was  expected  that  this  information 


TABLE  12 


LMET  GRADUATE  SELF-REPORT  ON  HANDLING  NEWLY  REPORTED  PERSONNEL 


1  2  3  4  5 

[Nothing  done  [Administrative  [Full  welcome 


Officer1 

personal ly] 

0 

3 

business  only] 

4 

3 

aboard  brief  & 
interview] 

42 

CPO 

1 

1 

3 

73 

5 

P01 

24 

1 

65 

2 

7  6 

OVERALL: 

3 

5 

13 

12 

16 

iTwo  officers  not  asked  this  question 

2" _ Royal  Red  Carpet  Treatment."  -  Division  Officer 

" _ Like  to  find  out  what  they  expect  from  the  Navy  to  help  satisfy 

their  need."  -  Division  Officer 

3" I  don't  want  him  to  do  the  shit  jobs  right  off  the  bat."  -  CPO 
"I  let  him  play  pigeon  for  a  few  days.”  -  CPO 

4" I  just  put  them  straight  to  work.... the  first  day  they  come  to  me,  I 
give  them  a  chipping  hammer  and  a  wire  brush  and  away  they  go."  -  P01 

5" I  see  all  personnel  for  administrative  purpose  only."  -  P01 

6" _ I  guess  I  don't  really  tell  him  what  I  expect... I  don't  express  it.. 

I  assume  he  will  talk  to  his  co-workers."  -  P01 
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would  be  picked  up  from  peers.  This  idea  was  best  stated  by  an  LMET 

first  class  petty  officer,  "I  guess  I  don't  really  tell  him  what  I 

expect... I  don't  express  it... I  assume  he  will  talk  to  his  co-workers." 

Aside  from  the  reporting  aboard  interview,  LMET  graduates 

expressed  a  wide  variety  of  methods  for  handling  newly  reported 

personnel  as  indicated  in  the  following  interview  remarks: 

I  don't  want  him  to  do  the  shit  jobs  right  off 
the  bat. . .CPO 

...let  him  play  pigeon  [running-mate  concept]  for 
a  few  days  -  P01 

...I  like  to  find  out  what  he  expects  from  the 
Navy  to  help  satisfy  his  needs  -  Div  Off 

...I  just  put  him  straight  to  work... The  first  day 
they  come  to  me  I  give  them  a  chipping  hammer  and 
a  wire  brush  and  away  they  go  -  P01 

...I  give  them  the  royal  red  carpet  treatment  for 
a  few  days  -  Div  Off 

The  LMET  graduates  that  tended  to  place  less  emphasis  on  smoothly 
integrating  personnel  into  their  work  groups  more  consistently  were 
those  that  had  the  least  healthy  organizations, 
f.  LMET  Reinforcement 

Another  aspect  of  the  "results"  level  was  briefly 
discussed  earlier  within  the  knowledge-content  domain.  It  concerns  the 
LMET  graduates  unmindful  disregard  of  LMET  concepts  and  vocabulary  at 
the  beginning  of  the  individual  interviews  with  a  noticeable  improve¬ 
ment  as  the  interview  progressed.  It  appeared  that  the  graduate  had 
totally  forgotten  some  of  the  LMET  issues  and  language  of  the  training. 
As  Table  13  indicates,  61%  of  the  graduates  never  used  their  student 
journal  after  completing  the  LMET  training.  More  significantly,  only 
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TABLE  13 


LMET  GRADUATE'S  SELF-REPORT  ON  USE  OF  THE  STUOENT  JOURNAL 


The  initial  question  for  this  data  was,  "Have  you  ever  used  the 
student  journal."  In  some  cases  the  response  was  "Yes"  but  not  for 
reasons  dealing  with  the  LMET  competencies.  Yes  answers  reflect  both 
aspects. 


TES 

NO 

Officer 

71 

9 

Chief  Petty  Officer 

52 

12 

First  Class  Petty  Officer 

83 

JO 

TOTALS  [<]: 

20  [39] 

31  [61] 

^Of  these  seven,  4  used  it  for  reasons  related  to  LMET. 
2Of  these  five,  3  used  it  for  reasons  related  to  LMET. 
30f  these  eight,  7  used  it  for  reasons  related  to  LMET. 
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27 %  of  the  sample  actually  used  the  journal  as  a  reference  or  guide  to 
a  problem  they  were  encountering.  This  is  inspite  the  fact  that  the 
manual's  intended  use  is  as  a  guide  and  reference  list  for  the  LMET 
competencies.  Additionally,  it  is  designed  to  be  used  in  helping  to 
establish  and  track  self, improvement  goals  in  leadership  and  management 
development  for  each  graduate.  The  indication  seems  to  be  that  by  not 
actively  using  the  student  journal,  LMET  graduates  are  not  keeping  the 
compentencies  foremost  in  mind  as  tools  for  leadership  and  management. 

A  last  general  observation,  based  on  responses  to  the 
query  as  to  whether  they  had  anything  else  they  would  like  to  mention, 
was  that  if  a  command  explicitly  or  implicitly  discouraged  the  use  of 
the  LMET  situational  approach  to  leadership  and  management,  a  very  high 
level  of  frustration  developed  within  the  graduate.  Graduates  that 
expressed  having  experienced  this  problem  seemed  to  use  three  different 
techniques  to  cope  with  their  anxiety.  First,  there  were  those  who 
continued  as  taught  through  LMET  and  just  lived  with  the  frustration. 
This,  of  course,  led  to  high  dissatisfaction  towards  their  job  and 
sometimes  created  conflicts  within  their  work  groups.  The  second 
approach  seemed  to  be  that  of  doing  the  exact  opposite  of  what  LMET 
taught.  For  example,  using  the  authoritarian  approach  to  its  extreme. 
Interestingly,  these  graduates  viewed  themselves  as  using  the  "old 
Navy"  approach  and  felt  they  were  completely  in  line  with  recent  CNO 
policy  statements  concerning  "pride  and  professionalism".  The  third 
method  seemed  to  be  the  development  of  a  more  passive  existence  through 
the  removal  of  oneself  from  the  active  chain  of  command,  becoming  more 
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of  a  bystander  [especially  in  one  instance  with  an  LMET  chief  petty 
officer].  Apparently  the  graduate  felt  a  loyalty  towards  the  LMET 
training  but  could  not  handle  a  situation  when  the  supervisor  did  not 
exhibit  any  support.  Again,  these  were  exceptions  to  most  graduates 
experiences  as  expressed  in  the  interviews,  but  may  nonetheless 
indicate  a  strong  need  for  external  support  mechanisms  in  order  for 
LMET  to  succeed. 

C.  SUMMARY  OF  LMET  FINDINGS 

Again,  the  findings  concerning  LMET  must  be  strictly  viewed  within 
the  context  of  the  limitations  of  a  non-random  sample.  The  issues 
Identified  and  the  points  raised  should  be  considered  hypotheses  which 
need  to  be  resolved  through  either  further  study,  or  within  a  more 
complete  full-scale  evaluation  to  test  their  actual  validity.  However, 
the  following  general  comments,  while  not  all  inclusive,  are  felt  to  be 
indicative  of  the  pilot  study  sample  and  are  therefore  included  as  a 
convenient  summary. 

Reactions  of  personnel  interviewed  concerning  LMET  were  upbeat, 
with  the  majority  of  respondents  in  agreement  as  to  its  necessity  and 
usefulness.  LMET  was  viewed  as  a  positive  experience  which  seems  to 
have  affected  graduates  in  differing  degrees.  The  actual  knowledge  of 
LMET  training,  while  not  always  initially  evident,  appeared  good. 
While  systematic  behavioral  changes  linked  directly  to  LMET  were  not 
uncovered,  there  were  specific  examples  and  indications  that  behavioral 
changes  had  taken  place. 
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LMET  program  strengths  such  as  graduate  increased  awareness,  the 
fulfillment  of  manager's  training  needs  in  the  areas  of  management/ 

leadership,  developing  a  positive  framework  for  healthy  work  groups, 

among  others  were  notably  in  evidence.  Several  areas  of  concern  were 

identified  and  require  further  investigation  to  determine  their 

validity.  Among  these  issues  were  the  following  four: 

Communication  Aspects  of  Rewarding  and  Disciplining 
The  Setting  of  Goals  and  Performance  Standards 
Feedback  Concerning  Performance 
The  LMET  Student  Journal 

It  is  unclear  to  the  extent  in  which  the  LMET  pilot  study  findings 
are  general izable  to  the  entire  population.  They  nevertheless  re. 
present  new  avenues  to  explore  in  developing  a  comprehensive  evaluation 
plan  for  LMET. 
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VI.  RECOMMENDATIONS  AND  CONCLUSIONS 


This  final  chapter  contains  the  conclusions  concerning  the  useful¬ 
ness  of  the  pilot  study  concept  as  well  as  recommendations  concerning  a 
future  overall  evaluation  of  the  LMET  program.  As  such,  the  chapter 
emphasizes  an  overview  of  three  general  areas  of  concern;  the  pilot 
study  concept,  particularly  as  applied  to  the  evaluation  of  LMET;  the 
interview  as  a  data  gathering  methodology;  and  relevant  issues 
requiring  further  investigation  in  developing  an  evaluation  plan  for 
the  Navy's  LMET  program.  These  three  areas  of  concern  are  divided  into 
two  sections  for  presentation;  first,  the  pilot  study  conclusions  and 
second,  the  recommendat ions  concerning  an  LMET  evaluation. 

A.  PILOT  STUDY  CONCLUSIONS 

The  central  theme  throughout  this  research  has  been  the  pilot 
study  concept.  Although  previously  stated,  it  is  of  sufficient  import¬ 
ance  to  re-emphasize  that  the  data  sample  for  this  study  was  neither 
random  nor  representative  of  the  Navy's  LMET  graduates.  The  51  LMET 
graduates  that  were  interviewed  contained  a  cross-section  of  various 
operational  commands,  billets,  rates,  ranks,  warfare  specialties  and 
geographic  locations  from  the  west  coast.  However,  the  data  was  suffi¬ 
cient  for  this  pilot  study  in  attempting  to  identify  Issues,  test 
hypotheses,  and  evaluate  a  methodology  pertaining  to  the  future  overall 
evaluation  of  a  major  Navy  program  such  as  LMET.  In  no  way  has  this 
pilot  study  eliminated  the  need  for  an  LMET  evaluation,  rather  it  rein¬ 
forces  that  need. 
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An  important  question  that  remains  unanswered  and  requires  further 
examination  is,  how  should  effectiveness  of  LMET  or  any  similar ]y  large 
program  be  measured?  This  pilot  study  concentrated  on  individual 
behavioral  changes  in  leadership/management  after  LMET  graduation. 
Other  effectiveness  criteria  for  the  LMET  program  could  include,  but 
are  not  limited  to:  individual  command  effectiveness,  reduction  of 
selected  Navy-wide  problems  or  concerns  [e.g.  retention,  substance 
abuse,  etc.],  and  knowledge/behavior  of  graduates  6  months  after  LMET 
training.  The  underlying  issue  is  determining  what  is  the  actual 
purpose  or  goal  of  the  Navy's  LMET  program.  Once  that  is  clearly 
defined,  then  an  evaluation  of  its  progress  towards  achieving  that  goal 
can  more  easily  be  conducted. 

The  continued  emphasis  on  the  limitations  of  this  pilot  study  is 
not  intended  to  diminish  its  usefulness  to  LMET  program  managers  and 
decision  makers.  Rather,  it  is  to  keep  the  pilot  study  results  within 
their  proper  perspective,  that  of  providing  preliminary  guidance 
towards  a  full-scale  LMET  evaluation.  This  study  did  examine  the  LMET 
training  with  the  stated  objective  of  providing  substantive  recommenda¬ 
tions  for  conducting  a  full-scale  evaluation  of  the  program.  The 
beneficial  accomplishments  derived  from  the  pilot  study  concept  as  re¬ 
lated  to  general  program  evaluation  fall  into  four  categories; 
efficiency,  effectiveness,  definative  results,  and  participant  educa¬ 
tion  and  learning. 
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1 .  Efficiency 

The  first  accomplishment,  that  of  efficiency,  can  be  ex¬ 
amined  from  three  different  resource  perspectives:  minimum  manpower, 
low  cost,  and  short  duration.  This  pilot  study  was  conducted  by  two 
lieutenant  commander.  Naval  Postgraduate  School  students  in  working  on 
their  master's  degree  thesis.  Their  time  was  already  committed  to  some 
form  of  thesis  work,  and  in  undertaking  a  pilot  study  of  the  Navy's 
LMET  program,  they,  as  well  as  the  Navy,  were  able  to  benefit  in 
"killing  two  birds  with  one  stone".  While  it  may  not  always  be  easy  to 
find  such  an  efficient  situation,  the  fact  of  the  matter  is  that  two 
Navy  personnel  conducted  the  study,  which  underscores  the  efficiency  of 
this  approach.  Further  emphasizing  efficiency  is  the  pilot  study's  re¬ 
latively  low  cost.  Only  $10,000  was  allocated  for  this  research, 
which,  when  compared  to  an  overall  program  evaluation,  seems  well 
worthwhile  to  insure  the  usefulness  of  the  full  evaluation.  Finally, 
there  is  the  timeliness  aspect  of  the  pilot  study,  6  months  from  start 
to  finish.  It  should  be  noted  that  this  was  accomplished  in  addition 
to  regular  ongoing  classes  and  studies. 

2.  Effectiveness 

The  next  accomplishment  of  the  pilot  study  is  in  the  area  of 
effectiveness.  When  this  pilot  study  commenced,  the  problem  that  was 
most  difficult  to  acertain  was  how  to  measure  effectiveness  of  such  a 
large  program.  It  was  not  clear  at  the  outset  exactly  what  to  measure 
or  how.  The  flexibility  afforded  by  only  two  people 
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undertaking  such  a  study  was  another  primary  cost  savings.  As  issues 
were  identified,  questions  developed,  and  the  evaluation  method 
formalized,  the  two  researchers  were  able  to  easily  adjust  and  proceed 
with  few  problems.  This  very  flexibility  permitted  the  gathering  of  an 
enormous  amount  of  high  quality  information  on  a  complex,  dynamic  Navy¬ 
wide  system. 

3.  Definitive  Results 

The  third  major  accomplishment  of  this  pilot  study  is  that  it 
produced  definitive  results.  It  did  develop  issues  and  recommendations 
on  how  to  conduct  a  full-scale  evaluation  of  the  LMET  program.  These 
findings,  issues,  and  recommendations  can  be  used  by  program  managers 
and  decision  makers  in  developing  a  plan  for  the  further  evaluation  of 
this  program  with  a  certain  amount  of  assurance  that  they  are  indeed 
proceeding  in  a  relevant  direction. 

4.  Education  and  Learning 

The  final  accomplishment,  that  of  education  and  learning, 
while  not  an  intended  goal,  further  establishes  the  value  of  a  pilot 
study.  In  order  to  develop  a  pilot  study  design  for  the  LMET  program, 
it  was  necessary  to  first  learn  about  the  program.  The  background  re¬ 
search  conducted  resulted  in  insights  into  the  overall  LMET  program, 
from  theoretical  conception  to  development  through  implementation. 
Additionally,  a  feeling  for  and  an  understanding  of  the  program,  as  it 
is  now  administered,  was  gained  by  the  researchers.  The  implication  is 
that  these  two  researchers  may  be  as  knowledgeable,  if  not  more  so, 
than  the  actual  program  managers  of  LMET.  This  educational  feature  of 
the  pilot  study  process  is  a  prime  argument  for  program  managers  to 
become  Intimately  involved. 
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5.  Future  Pilot  Studies 


With  these  four  considerations  in  mind  let  us  now  briefly 
examine  the  future  potential  of  pilot  studies  within  the  Navy.  On  the 
surface  it  appears  that  the  benefits  far  outweigh  the  costs  of  a  pilot 
study  completed  prior  to  a  comprehensive  program  evaluation.  If  more 
program  evaluations  were  designed  with  a  pilot  study  as  their  first 
step,  program  managers  by  involving  themselves,  could  feel  more  confid¬ 
ent  in  directing  the  full “scale  evaluation.  A  pilot  study  can  examine 
relevant  issues  such  as  who,  how  and  what  to  evalute.  Program 
managers  using  this  approach  can  gain  valuable  insight  and  direction 
for  their  program  evaluations,  thereby  making  better  decisions.  The 
pilot  study  concept  can  be  viewed  as  another  tool  that  managers  and 
decision  makers  can  use  to  improve  the  quality  and  timeliness  of  their 
decisions. 

8.  RECOMMENDATIONS  CONCERNING  LMET  EVALUATION 

This  section  addresses  some  definitive  recommendations  concerning 
a  future  full-scale  evaluation  of  LMET.  These  recommendations  are 
organized  to  answer  four  central  questions  concerning  that  evaluation: 
who?,  where?,  how?,  and  what?  While  not  all  inclusive,  it  is  felt  that 
these  insights  will  provide  sufficient  guidance  to  LMET  program  mana¬ 
gers  to  develop  a  useful  evaluation  plan. 

1.  Who 

The  first  question  concerning  the  evaluation  plan  is  who,  and 

can  be  further  divided  into  four  topical  areas;  the  individual  LMET 
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graduate,  the  chain  of  command,  operational  commands,  and  evaluators. 
All  of  these  areas  when  viewed  together  shed  light  on  the  who  question, 
a.  LMET  Graduates 

In  order  to  evaluate  the  effectiveness  of  the  LMET  program 
it  is  necessary  to  collect  data  from  those  individuals  most  affected  by 
the  training,  which  in  this  case  are  the  graduates.  Behavior  modifi¬ 
cation  or  improvement  in  the  area  of  leadership/management  is  one  of 
the  goals  of  the  program.  In  order  to  collect  relevant  data  which 
reflects  the  extent  of  this  improvement  [as  well  as  a  possible  cause 
and  effect  relation  to  the  training],  a  sufficient  amount  of  time  after 
graduation  must  have  elapsed  in  order  for  the  graduate  to  have  develop¬ 
ed  his  operating  style.  It  is  recommended  that  data  be  collected  from 
LMET  graduates  who  have  been  out  of  LMET  school  for  a  minimum  of  6 
months.  However,  as  time  increases  from  graduation,  a  cause  and  effect 
relationship  may  become  contaminated  due  to  other  factors.  In  addi¬ 
tion,  due  to  the  multi-levels  of  training  as  well  as  the  number  of 
graduates,  data  collection  should  concentrate  specifically  on  middle 
managers,  such  as:  department  heads,  division  officers,  chief  petty 
officers  [CPO]  and  first  class  petty  officers  [P01].  In  comparing  the 
responses  of  department  heads/CPO's  with  division  officers/POl ’s  the 
issue  of  seniority  as  compared  to  changes  in  behavior  can  be  further 
explored.  Realizing  that  data  collected  from  graduates  [self-report] 
may  be  suspect,  it  is  further  recommended  that  a  validation  procedure 
be  implemented.  The  validation  process  used  in  the  pilot  study  of 
collecting  additional  data  on  the  graduate  from  his  immediate  super¬ 
visor  and  subordinate  would  work  quite  well. 
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b.  Chain  of  Command 

An  issue  that  surfaced  as  a  result  of  the  pilot  study,  was 
the  effect  that  the  chain  of  command  had  on  a  graduate  towards  applying 
the  training.  By  collecting  data  on  the  entire  chain  of  command,  the 
effect  of  command  support  for  the  graduate  can  be  further  explored. 
Additionally,  as  time  permits,  indications  of  whether  the  graduate  is 
considered  an  average  or  superior  performer  should  be  sought  to  assist 
in  confirming  the  graduates  self-reported  information,  as  well  as  a 
first  step  in  revalidating  the  competencies  on  a  hierarchical  basis. 

c.  Operational  Commands 

Since  the  LMET  program  has  been  initially  directed  at 
operational  commands,  the  greatest  impact,  hence  potential  for  change, 
would  be  expected  to  occur  there.  As  such,  operational  commands  should 
be  the  specific  targets  of  the  evaluation.  One  additional  considera¬ 
tion  is  in  the  choice  of  which  graduates  to  specifically  emphasize. 
Since  leadership/management  behavior  [change  or  improvement]  is  the 
emphasis  of  the  evaluation,  only  graduates  in  this  role  should  be  con¬ 
sidered.  Air  squadrons  for  example,  have  many  junior  officer  [ENS, 
LTjg]  graduates  who  do  not  hold  active  leadership/management  positions 
and  would  therefore  provide  little  substantive  data. 

d.  Evaluators 

The  final  issue  is  who  should  conduct  the  evaluation  - 
civilians  or  Navy  personnel?  Constraints  of  sample  size,  time  and  man¬ 
power  availability  may  be  the  deciding  factors  of  actually  who  will 
conduct  the  evaluation,  civilian  or  military  personnel.  However,  one 
issue  is  clear,  that  the  program  managers  should  take  an  active  role 
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in  order  to  educate  themselves  on  the  LMET  program.  The  issue  of 
whether  to  employ  civilian  or  military  evaluators  hinges  to  a  great 
extent  on  the  data  collection  method  used.  If  interviewing.  Navy 
personnel  may  have  an  edge,  as  insiders  to  the  organization.  However, 
properly  trained  civilians,  skilled  in  areas  of  behavioral  psycology, 
may  be  able  to  quickly  overcome  that  advantage  due  to  their  insights  on 
human  behavior.  Additionally,  Navy  personnel  may  be  more  willing  to 
express  negative  opinions  on  the  LMET  program  to  civilians.  Obviously, 
using  both  types  of  personnel,  exploiting  their  strengths  would  most 
likely  provide  "the  best  of  both  worlds". 

2.  Where 

The  next  major  question  to  be  addressed  is  where  to  conduct 
the  evaluation.  To  reinforce  an  issue  discussed  earlier,  the 
evaluation  should  not  be  conducted  at  the  LMET  school.  Here  graduates 
will  most  likely  provide  insights  of  their  knowledge  and  not  behavior. 
Onboard  individual  commands  appears  to  be  the  most  ideal  location  to 
gather  behavioral  information.  In  this  way  the  respondents  can  remain 
within  familiar,  more  comfortable  [psychologically]  surroundings.  It 
is  also  less  disruptive  to  individual  command's  work  schedules,  hence 
more  likely  to  gain  their  acceptance  and  support.  The  specific  concern 
is  that  it  be  conducted  in  a  neutral  zone  within  the  command  where  the 
respondent  will  be  less  pressured  by  the  internal  environment.  One 
issue  that  did  surface  during  the  pilot  study  is  that  of  the  command's 
environment.  A  cross-section  of  environments  should  be  included  with¬ 
in  the  evaluation  sample.  There  was  an  effort  during  the  pilot  study 
to  concentrate  on  commands  with  easy  access  [i.e.  inport  within  the  3 
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geographical  areas],  hence,  a  tendency  to  evaluate  graduates  on 
commands  in  the  shipyards,  undergoing  heavy  inport  maintenance  programs 
or  training  programs.  Commands  in  all  phases  of  employment  should  be 
considered  such  as  in  shipyards,  in  homeport,  deployed,  during  overseas 
work.up  exercises,  and  on  overseas  standdowns,  so  as  to  not  bias  the 
data  due  to  any  unknown  external  environmental  effects. 

3.  How 

The  third  question,  that  of  how,  concerns  the  methodology  to 
use  in  conducting  the  evaluation.  This  section  is  further  divided  into 
three  areas  of  concern  which  are:  use  of  interviews,  use  of  surveys, 
and  mechanics  of  interviewing. 

a.  Use  of  Interviews 

The  pilot  study  used  interviews  as  the  primary  data 
collection  method.  It  was  found  to  be  an  effective  means  of  gathering 
data  concerning  non.quantifiable,  behavioral. oriented  responses.  The 
LMET  Stimulus  Questions  provided  by  McBer  and  Company  were  particularly 
useful  in  exploring  these  areas.  When  the  desired  data  is  directed  to. 
wards  actual  behavior  with  the  multitude  and  variety  of  responses 
possible,  interviewing  seems  to  be  the  best  method.  While  straight 
forward  questions  seeking  limited  responses,  factual  and  quantifiable 
data  are  more  efficiently  collected  using  methods  other  than 
interviews. 

b.  Use  of  Surveys 

Surveys,  as  opposed  to  interviews,  are  better  suited 
toward  easily  quantifiable  reponses,  factual  data,  and  limited 
responses.  Straight  forward  questions  that  are  less  affected  by  the 
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respondents  personal  opinions  a~.d  more  directed  at  specific  answers  and 
ranges  of  occurrences  are  more  efficiently  contained  in  a  survey.  The 
survey  can  encompass  a  larger  sampie  size  quickly  within  the 
limitations  previously  disscussed  in  earlier  chapters.  McBer's 
"Management  Behavior  Survey"  [partially  contained  in  Table  14]  is  an 
example  of  how  to  gather  data  on  the  graduate's  usage  of  LMET 
competencies.  When  asking  straight  forward  questions  such  as  these  on 
the  competencies,  the  resulting  data  is  sufficient  for  a  broad  general 
overview  of  implications.  If  such  a  survey  is  administered  to  only  the 
graduates,  it  takes  on  the  characteristics  of  an  unvalidated 
self-report  and  may  therefore  indicate  knowledge  and  opinion  rather 
than  actual  behavior.  Through  a  validation  process  with  supervisor  and 
subordinate  input,  such  a  survey  could  provide  insights  into  a 
graduates  behavior. 

c.  Mechanics  of  Interviewing 

Since  interviewing  was  the  primary  data  gathering  method 
for  this  pilot  study,  it  is  appropriate  to  mention  some  of  the  lessons 
learned  to  assist  in  using  this  method  within  a  full-scale  evaluation. 
The  presentation  is  sub-divided  into  four  general  areas  of:  training, 
interview  introduction,  methodology  and  tools. 

First,  the  training  necessary  to  successfully  conduct  in¬ 
terviews  is  minimal.  While  there  are  numerous  references,  R.  L. 
Gorden ' s  book  Interviewing:  Strategy,  Techniques  and  Tactics  prov ides  a 
useful  background  on  interviewing.  A  seminar  or  workshop  on  inter¬ 
viewing  as  applied  to  the  LMET  evaluation  process  should  provide 
interviewers  with  sufficient  skills  to  grasp  the  basic  techniques.  With 
additional  practice  and  critiquing  most  individuals  should  be  able  to 
develop  into  skilled  Interviewers. 
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TABLE  14 


SAMPLE  ITEMS  FOR  MANAGEMENT  BEHAVIOR  QUESTIONNAIRE 


Circle  the  number  which  best  represents  where  the  ratee's  behavior 
falls  relative  to  the  two  statements  provided.  [For  example,  circling 
2  would  indicate  a  greater  tendency  to  behave  in  accordance  with  the 
statement  on  the  left;  circling  ±  would  indicate  that  both  statements 
are  equally  descriptive  of  his  or  her  behavior.] 


Sets  goals  and  performance  standards: 


1.  He/She  sets  clear  specific 
goals. 

1  2  3 

2.  He/She  will  change  the  goals 
to  make  them  realistic. 


1  2  3 

3.  He/She  sets  milestones  when 
assigning  a  task  and  checks 
to  see  if  things  are  on 
schedule. 


Takes  Initiative: 

4.  He/She  initiates  action 
rather  than  waiting  to 
react  as  situations  develop. 

1  2  3 

5.  He/She  experiments  with 
and  introduces  innovative 
work  procedures. 

1  2  3 

6.  He/She  spends  unusually  long 
hours  on  tasks  when  they 
require  it,  whether,  he/she 
is  asked  to  do  so  or  not. 

1  2  3 


He/She  sets  vague  goals. 


4  5  6  7 

He/She  will  keep  trying  to 
reach  a  qoal  even  if  it  does 
not  appear  realistic. 

4  5  6  7 

He/She  gives  out  a  task  and 
checks  back  about  the  time  it 
should  be  done. 


4  5  6  7 


He/She  takes  appropriate 
actions  to  handle  situations  as 
they  develop. 

4  5  6  7 

He/She  uses  traditional,  well- 
established  work  procedures. 


4  5  6  7 

He/She  will  work  regular  hours 
on  a  task  and  if  It  takes 
longer  than  that  to  complete 
will  come  back  to  it  later. 

4  5  6  7 
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Second,  the  introduction  to  the  interview  is  a  key  to 
establishing  good  rapport  with  the  respondent,  thereby  ensuring  a  good 
flow  of  information.  A  good  introduction  needs  to  stress  the 
conf identiality  of  the  interview  and  emphasize  that  its  overall  intent 
is  to  evaluate  a  program  and  not  the  individual.  Putting  the  respond¬ 
ent  at  ease  initially  is  important  and  can  be  assisted  by  beginning  the 
interview  with  easier  "warm-up"  questions.  A  final  point,  is  that  most 
groups  of  related  questions  have  a  natural  flow,  which  should  be 
identified  and  followed.  This  will  assist  in  guiding  the  interview 
process  and  in  fact  eliminate  the  need  to  ask  certain  questions  as  they 
may  be  answered  before  they  arise. 

The  next  issue  is  that  of  the  interview  method,  and 
specifically  concerns  interviewer/respondent  interaction.  The  purpose 
of  the  interviewer  is  to  guide  the  conversation  in  the  desired 
direction  without  stifiling  the  information  flow.  Respondents  will 
normally  use  the  interview  as  a  means  to  voice  their  opinion  in 
unrelated  areas,  hence  they  may  ramble  or  get  off  onto  tangents.  Firm 
direction  from  the  interviewer  can  keep  it  both  healthy  and  on  track. 
However,  one  should  not  get  so  locked  into  a  rigid  process  as  to  miss 
the  surfacing  of  relevant  issues.  These  areas  should  be  explored  fully 
to  uncover  any  new  areas  of  concern. 

The  final  area  concerns  the  interviewing  tools  available. 
A  variety  of  "tools"  are  available  for  conducting  interviews,  each  with 
definate  advantages  and  disadvantages.  The  final  decision  on  using  a 
tool  is  how  it  will  assist  in  the  interviewing  process.  If  it  does  not 


provide  advantages  or  specific  assistance  to  the  interviewer  its  use 
should  be  carefully  reconsidered.  Some  “tools"  available  for  use  by 
the  interviewer  include  tape  recorders  [should  have  tape  counter  for 
indexing],  coded  answer  sheets,  and  preformated  interview  schedules. 
This  last  item  was  found  to  be  a  valuable  tool  in  conducting  the  pilot 
study,  in  that  it  assisted  the  interviewers  in  standardizing  their 
daily  routine  while  providing  the  command  with  clear  guidance  as  to  how 
to  schedule  their  personnel. 

4.  What 

The  final  question,  that  of  what,  concerns  issues  which  should 
be  considered  in  an  evaluation  of  the  LMET  program.  The  pilot  study 
surfaced  certain  issues  which  require  further  investigation  in  the 
course  of  a  full-scale  evaluation.  These  findings  should  be  considered 
unvalidated  hypotheses  that  require  further  study  or  additional  testing 
within  an  overall  evaluation  process.  There  are  seven  major  issues 
that  will  be  discussed:  graduate's  knowledge  level,  increase  in 
self-awareness,  cause  and  effect  relationships,  communication  skills, 
use  of  the  journal,  retention,  and  reinforcement  of  behavior. 

a.  Level  of  LMET  Graduate's  Knowledge 

While  not  specifically  the  emphasis  of  the  pilot  study, 
certain  levels  of  knowledge  exhibited  by  LMET  graduates  became  evident. 
Further  evaluation  of  the  LMET  program  should  in  fact  try  to  determine 
the  level  or  degree  to  which  LMET  knowledge  is  retained  after  gradua¬ 
tion  by  examining  the  extent  to  which  their  vocabulary  and  concepts 
conforms  to  the  LMET  competency  based  model.  In  addition,  the  ability 
to  discuss  techniques  actually  practiced  can  be  compared  with  course 


instruction  to  provide  some  insight  into  how  well  the  competency  model 
is  understood.  This  data  may  also  provide  insight  as  to  the  effective¬ 
ness  of  the  classroom  instruction  at  the  school. 

*  b.  LMET  Graduate  Self-Awareness 

The  pilot  study  data  do  indicate  that  graduates  have  a 
heightened  sense  of  awareness  on  how  their  actions  effect  others  as 
well  as  how  the  actions  of  others  effect  them.  This  issue  deserves 
further  investigation,  in  that  this  increased  self-awareness  could  be 
the  first  step  in  a  more  definative  behavior  change  in  leadership  and 
mangement  style.  While  admittedly  a  self-report,  if  it  is  supported  by 
indications  of  a  behavioral  change  then  LMET  should  be  credited  for 
beginning  the  trend  towards  modification  of  leadership/management 
skills  towards  the  competency  based  model. 

c.  Cause  and  Effect  Relationships 

The  pilot  study  data  do  not  support  any  systematic  cause 
and  effect  relationship  between  LMET  graduates  and  increased 
performance.  There  were,  however,  at  least  two  cases  [validated  by 
supervisor  and  subordinate]  where  the  LMET  training  resulted  in  a 
complete  and  highly  visible  turnaround  in  a  managers  behavior  and 
performance.  However,  both  cases  involved  managers  who  were  considered 
marginal  performers  and  were  sent  to  the  school  as  a  last  improvement 
effort.  The  Important  Issue  that  remains  to  be  determined  is:  does 
LMET  school  improve  leadership/management  behaviors,  hence  Increase 
managerial  performance?  A  side  Issue  seems  clear,  in  that  no  cases 
were  uncovered  showing  any  decrease  In  performance  after  LMET  school. 
There  appears  to  be  a  definate  need  for  further  evaluation  on  this 


subject,  to  determine  if  LMET  training  alone  can  bring  about  managerial 
behavior  change  or  improvement  resulting  in  increased  performance.  If 
LMET  school  itself  is  not  sufficient,  what  if  any  additional  support  is 
needed? 

d.  LMET  Graduate  Communication  Skills 

LMET  graduates  overall  demonstrated  a  noticeable  weakness 
in  the  use  of  communication  skills  and  techniques  in  their  leadership/ 
management  roles,  as  supported  by  the  pilot  study  data.  The  issue  of 
communication  skills  needs  to  be  further  investigated  in  order  to 
identify  specific  areas  that  may  need  additional  emphasis  during  the 
LMET  training.  Of  the  sixteen  LMET  taught  competencies,  four  were 
identified  as  specific  concerns  from  the  data.  These  were:  a  lack 
among  LMET  graduates  in  setting  performance  standards;  a  lack  of  goal 
setting,  both  personal  and  workgroup;  limited  use  of  rewards  and 
disciplines,  coupled  with  a  lack  of  flexibility  for  different 
situations;  and  the  use  of  advising  and  counseling  for  correcting  poor 
performance  only.  Each  of  these  specific  competencies  can  be  treated 
as  a  seperate  issue  for  further  examination  and  evaluation.  However, 
they  should  be  recognized  as  related  by  a  common  thread  as  forms  of 
managerial  communication. 

The  perception  that  LMET  graduates  seem  unclear  or  do  not 
know  what  is  expected  [performance]  of  them,  as  well  as  their 
subordinates'  similar  concerns,  may  have  implications  beyond  LMET.  This 
hypotheses  should  be  a  first  priority  to  test,  using  either  additional 
research  or  in  the  evaluation  plan.  While  it  may  only  be  conjecture, 
this  and  other  communications  concerns  may  well  hold  a  key  to  increased 
performance  and  satisfaction  within  Navy  commands. 
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e.  Graduate  Use  of  LMET  Student  Journal 

This  issue  is  two-fold  in  nature  depending  upon  the 
determination  of  the  first  concern,  which  is,  what  is  the  purpose  of 
the  student  journal?  LMET  designers  would  contend  that  the  student 
journal  is  a  tool  to  be  used  by  the  graduate  to  help  evaluate  and 
reinforce  actual  leadership  performance  and  self-made  goals  outside  of 
the  classroom.  Additionally,  it  could  serve  to  rearouse  the  LMET 
competencies.  As  such  it  should  reinforce  skills  and  behaviors  that 
exemplify  the  LMET  competencies  while  helping  to  correct  those  that  do 
not.  Dr.  Winter  further  explains  that  the  student  journal  and  its 
personal  student  log,  is  the  personal  written  record  of  an  individual's 
strengths  and  weaknesses,  set  against  the  Navy's  standard  competencies 
that  are  associated  with  excellence  and  superior  leadership 
performance. [WINTER,1979,pp. 11-12]  The  first  issue,  then  would  be,  to 
explore  the  purpose  of  the  LMET  student  journal.  If  it  is  determined 
to  be  a  beneficial  tool  to  be  used  by  the  graduate,  then,  the  second 
issue  would  be  to  determine  when,  why,  and  how  is  it  used.  The  data  do 
indicate  that  student  use  of  the  journal  is  minimal. 

f.  Retention 

The  issue  of  retention  is  of  obvious  importance  to  the 
Navy,  but  also  pertains  to  LMET.  This  issue  involves  two  underlying 
thoughts.  The  first  is,  does  LMET  training  have  any  effect  on  the 
graduate's  retention?  The  senority  aspect  of  this  may  involve  an 
examination  of  LMET's  effect  on  junior  personnel  [division  officers, 
and  first  class  petty  officers]  as  opposed  to  its  effect  on  more  senior 
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personnel  [chief  petty  officers  and  department  heads].  Should  further 
investigation  reveal  that  LMET  training  has  a  positive  impact  on  re¬ 
tention,  it  may  well  indicate  a  need  for  increased  emphasis  on  other 
levels  of  personnel.  The  second  thought  is  investigating  the  effect  of 
the  graduates  training  on  subordinates  in  relation  to  retention. 
Again,  if  further  examination  indicates  a  positive  impact  on  retention 
by  LMET  graduates,  it  would  assist  in  justifying  the  cost  of  the 
program. 

g.  Reinforcement  of  LMET  Competencies 

A  final  issue  is  the  need  for  reinforcement  of  LMET 
competencies  as  perceived  by  the  graduates.  The  pilot  study  data 
indicates  that  LMET  competencies  are  more  likely  to  be  practiced  when 
reinforced  within  the  command  [specifically  by  supervisors].  This 
issue  of  internal  reinforcement  was  also  evident  from  several  LMET 
graduates  who  commented  that  active  discouragement  of  LMET  "new  ideas" 
not  only  discouraged  them,  but  frustrated  their  efforts  in  leadership 
and  management.  Internal  reinforcement  of  the  LMET  competencies  and 
the  resultant  effect  on  the  graduate  should  be  furthered  explored. 

An  additional  related  area  requiring  research  is  in  the 
area  of  external  support  for  LMET  training.  This  support  or  lack  of 
support  outside  of  a  command  may  provide  some  insight  into  the  notion 
that  LMET  ends  upon  graduation  from  the  school.  This  hypothesis  could 
be  tested  during  the  full-scale  evaluation  as  to  whether  external 
support  or  lack  of  support  in  fact  affects  the  application  of  LMET 
competencies  by  the  graduate. 
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These  seven  issues,  while  not  all  inclusive,  should 

provide  sufficient  guidance  to  program  managers  and  decision  makers  to 
design  and  conduct  a  meaningful  evaluation  of  the  LMET  program. 

C.  CONCLUDING  REMARKS 

This  thesis  has  emphasized  the  central  point  that  the  pilot  study 
was  not  an  evaluation  of  the  LMET  program,  but  rather  the  preliminary 
step  to  overall  program  evaluation  to  determine  relevant  issues  and 
investigate  a  methodology  to  confront  them.  As  a  result,  it  has 

uncovered  issues  that  require  further  investigation  through  research  or 
within  a  comprehensive  full-scale  evaluation. 

Navy  leaders  and  managers  at  all  levels  have  a  real  need  to  know 
the  effectiveness  of  the  LMET  program,  that  is,  what  it  can  or  cannot 
do,  as  well  as  how  to  best  support  it.  In  times  of  austere  funding  and 

scarce  resources,  only  those  programs  that  can  demonstrate  their 

usefulness  should  justify  the  support  of  those  in  control.  The  clear 
way  to  demonstrate  this  usefulness  is  to  conduct  a  program  evaluation. 
This  can  identify  weaknesses  requiring  correction  as  well  as  strengths 
that  need  re-emphasizing  while  justifying  support  for  the  program. 

The  pilot  study  concept  is  a  useful  tool  for  program  managers  and 
decision  makers  to  employ  prior  to  committing  expensive  resources  to  an 
overall  evaluation.  The  measurement  of  effectiveness  is  at  best  a 
tenuous  business.  The  pilot  study  has  explored  this  issue  to  help 
develop  a  viable  method  to  address  effectiveness. 

The  flexibility  afforded  by  a  pilot  study  was  evident  in  identify¬ 
ing  relevant  Issues  which  were  unknown.  Insights  gained  on  the  LMET 


program,  as  the  pilot  study  progressed  were  used  to  update  and  modify 
its  direction.  In  this  way,  new  issues  were  explored  for  relevance 
almost  immediately.  A  large  scale  evaluation  would  have  found  these 
constant  changes  to  be  very  frustrating  and  costly.  For  the  pilot 
study  this  flexibility  was  gained  at  no  extra  cost.  The  bottom  line 
then  becomes  relevant  issue  identification  at  low  cost.  It  must  be 
remembered  however,  that  issues  identified  by  the  pilot  study  should  be 
considered  unvalidated  hypotheses  to  be  further  researched  and  explored 
or  perhaps  tested  within  an  overall  program  evaluation. 

A  final  aspect  of  the  pilot  study  which  cannot  be  over  emphasized, 
is  its  potential  as  a  learning  device  for  program  managers.  If 
involved  in  the  pilot  study,  program  managers  can  not  help  but  gain  a 
sharp  insight  into  their  programs.  This  insight  can  help  focus  their 
attention  onto  the  relevant  issues  that  need  to  be  addressed  in  a  full- 
scale  evaluation.  Furthermore,  the  knowledge  gained  by  program 
managers  can  make  it  easier  to  monitor  the  progress  of  the  larger 
evaluation. 

The  pilot  study  provides  a  method  whereby  program  managers  can 
quickly  and  efficiently  become  aware  of  relevant  issues.  More  import¬ 
antly,  it  can  be  used  to  provide  direction  and  guidance  for  a  meaning¬ 
ful  program  evaluation,  prior  to  committing  extensive  resources.  An 
effective  evaluation  strategy  is  the  key  to  healthy  program 
development. 
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APPENDIX  A 


LMET  GRADUATE  SELF-REPORT  OF  ACTIVITIES  AND  RESULTS23 


1.  How  long  ago  did  you  attend  LMET?  Did  you  attend  prior  to  the 
command? 

2.  Has  there  been  any  change  in  how  you  do  your  job,  how  you  lead  and 
manage  others,  after  taking  the  LMET  course?  [Probe  for  specific 
examples] 

3.  Has  there  been  any  change  in  your  own  career  or  career  plans  after 
taking  the  LMET  course?  [e.g.  promotion,  schools,  PQS,  retention 
intentions]  [Probe  for  specifics] 

4.  How  is  your  performance  on  your  job  measured?  [Probe  for  relevant 
indicies,  rewards,  task  accomplishment,  managerial  ability, 
professional  competence,  etc.]  Has  there  been  any  change  in  these 
job  performance  measures?  What  specifically? 

5.  Has  there  been  any  change  in  your  unit? 

-  In  job  performance  measures  (e.g.  inspections  passed,  producti¬ 
vity  indicies,  PMS  accomplishment  rate,  accidents,  etc.]. 

What  specifically? 

-  In  satisfaction  measures?  [e.g.  retention  rates,  NJP's,  sick 
calls,  UA's,  etc.] 

6.  What  goals  did  you  set  in  the  LMET  course?  How  have  you  completed 
these  goals?  Progress?  Action  steps  completed? 

7.  Have  you  ever  used  the  student  journal  since  you  left  the  course? 
For  what  reason?  [Probe  for  specifics] 

8.  What  was  your  level  of  participation  in  the  LMET  course  [low, 
medium,  high]? 

-  In  class 

-  Homework 

9.  Would  you  send  anyone  to  LMET  school? 


23The  LMET  graduate  was  asked  these  9  questions.  Questions  2-8  were 
provided  by  McBer  and  Company. 
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LMET  GRADUATE  STIMULUS  QUESTIONS 


24 


1.  When  you  are  faced  with  a  new  task,  how  do  you  go  about  accomplish, 
i  ng  i  t? 

2.  In  your  work  center,  how  do  you  assign  duties  and  responsibilities? 
What  kind  of  paperwork  do  you  do?  [In  relation  to  those  assign, 
ments] 

3.  What  type  of  roadblocks  do  you  run  into  on  your  job,  and  what  do  you 
do  about  them? 


4.  What  do  you  do  if  the  work  of  your  subordinates  is  worse  than  you 
expected? 

5.  What  do  you  do  if  the  work  of  your  subordinates  is  better  than  you 
expected? 


6.  How  do  you  go  about  getting  someone  to  buy  your  idea  about  a  better 
way  of  doing  things? 

7.  Tell  me  of  a  time  when  you  identified  a  problem  before  anyone  else 
did?  How  did  you  do  this? 


8.  How  do  you  go  about  getting  someone  settled  in  who  is  new  to  the 
work  center?  [What  is  expected  of  him?] 

9.  What  happens  to  your  work  center  when  you  are  not  there?  If  you 
took  two  weeks  of  emergency  leave  what  would  you  expect  to  find  upon 
your  return?  Why? 

10.  How  do  you  keep  cool  when  someone  is  pulling  your  chain? 


2^The  LMET  graduate  was  asked  these  10  questions.  All  questions 
were  provided  by  McBer  and  Company  with  the  exception  of  part  2  to 
question  9. 
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SUPERVISOR  QUESTIONS  VALIDATING  LMET  GRAOUATE'S  ACTIVITIES 

AND  RESULTS  25 


1.  Are  you  an  LMET  graduate?  How  long  ago  did  you  graduate? 

2.  How  would  you  describe  the  way  in  which  this  subordinate  [division 
officer,  leading  chief,  work  center  supervisor]  leads  and  manages 
others?  Could  you  give  me  specific  examples? 

3.  Has  there  been  any  change  in  your  subordinates  career  [e.g. 
promotions,  schools,  PQS,  accomplishments]  or  career  plans  [e.g. 
re-enlistment,  extensions,  service  obligations]  while  under  the 

supervision,  leadership  and  management  of  this  _ [division  officer, 

leading  chief,  work  center  supervisor]?  What  specifically?  What 
has  he  done? 

4.  How  is  your  subordinate's  [division  officer,  leading  chief,  work 
center  supervisor]  performance  measured?  [Probe  for  specifics] 

Have  you  changed  any  of  these  measures? 

5.  Has  there  been  any  changes  in  your  unit  as  related  to: 

-  job  performance  measures  [e.g.  inspections,  productivity, 
accidents,  PMS  accomplishments] 

-  satisfaction  measures  [e.g.  retention  rates,  NJP's,  sick  call, 
absent  ism]  What,  specifically?  [Probe  for  examples] 

6.  What  do  you  like  or  admire  most  about  his  performance  onboard? 

Least?  [Probe  for  specifics] 

7.  Would  you  work  for  him?  Why  or  why  not? 

8.  What  would  you  change  in  him  if  you  could?  Why? 

9.  Would  you  send  anyone  to  LMET  school?  Why? 

10.  Is  there  anything  else  you  might  like  to  add  in  regards  to  this 
individual  or  LMET? 


25The  supervisor  to  the  LMET  graduate  was  asked  those  10 
questions.  Questions  2-5  were  provided  by  McBer  and  Company. 
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SUBORDINATE  QUESTIONS  VALIDATING  LMET  GRADUATE'S  ACTIVITIES 

AND  RESULTS26 


1.  Are  you  a  LMET  graduate?  How  long  ago  did  you  graduate? 

2.  How  would  you  describe  the  way  in  which  your  boss  [division 
officer,  leading  chief,  work  center  supervisor]  leads  and 
manages  others?  Could  you  give  me  specific  examples? 

3.  Has  there  been  any  change  in  your  own  career  [e.g.  promotions, 
schools,  PQS  accomplishments]  or  career  plans  [e.g.  re. enlist¬ 
ment,  extensions,  service  obligations]  while  under  the  leader, 
ship  and  management  of  your  boss  [division  officer,  leading 
officer,  work  center  supervisor]?  What  specifically?  What  has 
he  done? 

4.  How  is  your  performance  on  the  job  measured  by  your  boss 
[division  officer,  leading  chief,  work  center  supervisor]? 
[Probe  for  specific  indicies]  Have  you  noticed  any  changes  in 
these  performance  measurements? 

5.  Has  there  been  any  changes  in  your  unit  as  related  to: 

.  job  performance  measures  [e.g.  inspections  passed, 
productivity,  accidents,  PMS  accomplishments] 

.  satisfaction  measures  [e.g.  retention  rates,  NJP's,  sick 
calls,  absentism]  What,  specifically?  [Probe  for  examples] 

6.  What  do  you  like  or  admire  most  about  his  performance  onboard? 
Least?  Can  you  give  specifics? 

7.  Would  you  work  for  him  again?  Why  or  why  not? 

8.  What  would  you  change  in  him  if  you  could?  Why? 

9.  Would  you  like  to  attend  LMET  school  if  you  could?  What  do  you 
think  you  would  get  out  of  it?  Why? 

10.  Is  there  anything  else  you  would  like  to  tell  me  in  regards  to 
your  boss  or  LMET? 


^6The  subordinate  to  the  LMET  graduate  was  asked  these  10 
questions.  Questions  2.5  were  provided  by  McBer  and  Company. 
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